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FOREWORD 


ScenPro  has  been  working  with  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  Infantry  Forces  Research  Unit  at  Fort  Benning  to  develop  and  field  the  Virtual 
Soldier  Skills  Assessment  (ViSSA)  system.  The  ViSSA  system  is  an  automated  training, 
assessment,  and  after-action  review  tool  for  virtual  simulations.  It  assists  the  Observer/Controller  in 
preparing  dismounted  forces  for  critical  decision-making  situations  during  military  operations  and 
providing  mission  rehearsal  for  operations  in  urban  environments.  ViSSA  tracks  mission-related 
factors  linked  to  Soldier  decisions,  movements,  rounds  fired,  contact  with  virtual  entities,  and  time 
factors  using  the  protocol  data  unit  (PDU)  information  packets  generated  by  entities  during  the 
simulation.  The  PDU  packets  provide  information  about  an  entity’s  status,  position,  heading,  and 
how  and  where  the  entity  should  be  displayed  in  the  virtual  environment.  The  key  research  thrust  of 
this  SBIR  was  to  investigate  ways  to  assess  Soldier  decision  making  skills  using  primarily  data 
available  via  the  PDU  packets.  The  result  is  a  training  and  analysis  system  that  can  accurately 
monitor  Soldier  performance  and  quickly  summarize  strengths  and  weaknesses,  as  defined  by  the 
trainer,  across  key  performance  dimensions.  Individual  project  goals  to  increase  leader 
responsiveness  to  environmental  stimuli,  situational  awareness,  and  mental  agility  to  make  sound 
leadership  decisions  are  incorporated  into  the  ViSSA  system  design.  These  characteristics  support 
concepts  for  the  Objective  Force  and  complement  the  Future  Combat  Systems  program. 

This  research  was  funded  by  ARI  as  a  Department  of  the  Army  Small  Business  Innovation 
Research  Phase  II  award  entitled  Assessing  Decision-Making  Skills  in  Virtual  Environments.  The 
final  results  from  the  Phase  II  effort  and  a  system  demonstration  were  provided  to  the  Chief  of  the 
Virtual  Simulation  Lab  and  key  staff  members  in  November  2003.  The  ViSSA  system  is  currently 
installed  and  operating  at  various  facilities  within  the  Soldier  Battle  Lab  at  Fort  Benning,  Georgia. 


MICHELLE  SAMS 
Technical  Director 
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DISMOUNTED  INFANTRY  DECISION  SKILLS  ASSESSMENT  IN  THE  VIRTUAL 
TRAINING  ENVIRONMENT 


EXECUTIVE  SUMMARY 


Research  Requirement: 

ScenPro,  Inc.  has  developed  the  Virtual  Soldier  Skills  Assessment  System  (ViSSA)  in 
cooperation  with  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
through  Phase  II  Small  Business  Innovation  Research  (SBIR)  funding.  ViSSA  is  an  automated 
training,  assessment,  and  after-action  review  support  system  designed  to  assist  in  preparing 
dismounted  forces  for  decision-making  in  urban  operations,  using  the  strengths  of  virtual 
environment  technologies. 

The  U.S.  Army  is  investigating  the  use  of  virtual  environment  (VE)  training.  Currently,  the  VE 
training  facilities  for  dismounted  infantry  are  envisioned  to  be  similar  to  the  Soldier  Battle  Lab 
(SBL)  located  in  Fort  Benning.  Due  to  the  complexity  of  the  VE,  observer/controllers  (O/C)  are 
forced  to  spread  their  attention  across  many  areas  during  an  exercise.  O/Cs  need  automated  support 
that  enhances  the  training  environment. 

The  ViSSA  prototype  system  was  designed  to  enhance  the  O/C’s  ability  to  monitor  and  train 
cognitive  decision-making  skills  through  the  execution  of  missions  by  small  unit  leaders  (platoon, 
squad,  and  team)  in  the  virtual  environment.  In  particular,  ViSSA  automatically  identifies 
significant  events,  logs  all  Soldier  movement  and  behavior,  and  provides  focus  during  after  action 
reviews  (AAR). 

Leader  trainees  command  a  platoon,  squad  or  fire  team  during  a  series  of  realistic  training 
missions  in  the  virtual  environment,  leveraging  the  safety  and  adaptability  offered  by  virtual 
technology  and  enhanced  by  ViSSA  assessment  capabilities  and  multi-media  after-action  reviews. 
The  result  is  a  comprehensive  training  system  that  assesses  decision-making  and  team  coordination 
skills,  yet  requires  a  low  level  of  personnel  support  for  O/C  functions,  subordinates,  and  role  players 
by  using  the  computer-controlled  or  semi-automated  forces  that  represent  subordinates,  friendly 
forces,  enemy  forces,  and  civilians. 

The  research  goals  for  this  effort  were  threefold:  1)  to  design  an  automated  prototype  system 
capable  of  accepting  predefined  assessment  criteria  from  the  O/C  during  the  staging  of  the  virtual 
exercise;  2)  to  develop  a  methodology  for  capturing  and  linking  key  leadership  decisions  to 
observable  commands,  infantry  tactics,  and/or  communications  established  by  the  virtual  training 
objectives;  and  3)  to  develop  a  means  of  displaying  decision-making  shortfalls  and  successful 
execution  of  training  objectives  in  real-time  and  immediately  following  a  training  exercise  to 
provide  pertinent  feedback  to  Soldiers  during  the  after-action  review. 

Procedure: 

Research  into  legacy  systems  literature  and  current  development  studies  provided  opportunities 
to  identify  compatible  virtual  environment  technologies  for  integration  and  future  development  and 
research.  The  ViSSA  development  team  observed,  tested,  and  participated  in  virtual  training 
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exercises  and  experiments  at  the  SBL  to  understand  the  state-of-the-art  in  virtual  environment 
technologies.  In  addition,  research  conducted  at  the  Close  Combat  Tactical  Trainer  facility  provided 
insights  and  understanding  into  the  interaction  of  computer-driven,  manned,  close  combat  heavy 
vehicle  simulators  (e.g.,  M1A1  tank  and  M2A2  Bradley  Fighting  Vehicle)  combined  with  computer 
generated  forces  operating  together  on  a  virtual  battlefield . 

In  order  to  fulfill  the  requirements,  the  OneSAF  Testbed  Baseline  (OTB)  software,  DISAF, 
Soldier  Visualization  Station  software,  ASTi™  Digital  Audio  Communication  System  (DACS), 
Distributed  Interactive  Simulation  (DIS)  protocol,  and  High  Level  Architecture  (HLA)  protocol, 
which  are  used  at  the  SBL,  were  evaluated  for  integration  with  the  SBL.  The  ViSSA  system  does 
not  attempt  to  replace  or  interfere  with  existing  software  components  providing  the  2-D  and  3-D 
graphical  representations  of  the  simulation.  Instead,  the  ViSSA  Assessment  module  monitors 
network  traffic,  internalizes  the  movement  and  behavior  of  the  Soldiers,  and  automatically  detects 
significant  events.  Concurrently,  the  ViSSA  Logger  module  captures  and  logs  all  entity  protocol 
data  units  (PDU)  packets  while  the  ViSSA  Audio  Logger  module  captures  and  logs  all  DACS-based 
communications  packets.  This  design  allows  ViSSA  to  operate  with  almost  any  virtual  environment 
software  that  communicates  via  (PDU)  packets  across  a  DIS/HLA  network  and  ensures  ease  of 
integration  as  virtual  environment  software  graphics  and  technologies  continue  to  evolve. 

Subject  matter  experts  (SMEs)  in  the  areas  of  training,  urban  warfare,  and  virtual  environment 
system  operation  participated  throughout  the  development  effort.  Contributions  from  the  SMEs 
pointed  towards  a  rule-based  expert  system  to  monitor  behavior  to  identify  significant  events  for 
AAR.  This  user-driven  approach  provided  a  firm  foundation  for  a  functional  system  design  that 
employs  a  flexible  schema  of  logic  for  implementing  assessment  rules  in  ViSSA.  The  schema  is 
known  as  Event/Condition/Action  (E/C/A)  and  is  a  variant  of  traditional  production  rules  that 
accommodates  both  the  real-time  and  reactive  requirements  of  the  simulation  environment.  Every 
action  and  behavior  of  an  entity  within  the  simulation  produces  an  Event.  Events  cause  blocks  of 
software  (VBScript)  containing  one  or  more  conditions  to  be  evaluated.  If  the  Conditions  are  met, 
then  the  resulting  Action  takes  place.  The  specific  Events  supported  by  ViSSA  include  movement, 
firing  a  weapon,  change  of  posture,  communication,  and  any  other  behavior  that  is  communicated 
across  the  DIS/HLA  network.  In  addition,  ViSSA  allows  the  0/C  to  predefine  observations  that 
cannot  be  detected  and  communicated  via  the  DIS  protocol.  The  system  is  capable  of  combining 
and  reasoning  over  both  computer  and  human  detected  events.  A  Condition  is  a  Boolean  expression 
triggered  by  the  Event.  Examples  of  this  include  checking  if  the  Soldier  entered  a  specific  building, 
ducked  under  a  window,  or  used  a  certain  type  of  weapon.  The  resulting  Action  can  be  as  simple  as 
directing  the  system  to  mark  a  point  on  the  exercise  timeline  for  later  review,  or  starting  a  timer  to 
measure  reaction  time  to  enemy  fire.  A  complete  example  of  E/C/A  would  be  the  following:  an 
Event  is  created  by  a  Soldier  firing  his  weapon,  the  Condition  is  met  by  targeting  an  Opposing 
Force,  and  the  resulting  Action  is  placing  a  mark  on  the  exercise  timeline  indicating  the  sequence 
should  be  reviewed  during  the  AAR. 

The  E/C/A  Assessment  module  of  ViSSA  is  built  by  combining  the  Microsoft  Visual  Basic™ 
Scripting  Engine  with  a  ViSSA  COM  Automation  Object  Model.  Because  the  existing  OTB  logger 
was  not  suitable,  a  Logger  module  was  developed  to  provide  many  features  that  go  beyond 
traditional  DIS  PDU  loggers  and  to  provide  integration  with  MS  Office  products.  Finally,  an  Audio 
Logger  was  developed  to  synchronize  the  audio  with  the  data  of  the  exercise.  Each  module  is 
designed  to  run  on  a  separate  mid-range  WinTel  computer  running  Windows  2000  (or  later). 
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Findings: 


Initial  capabilities  assessments  by  BattleMasters  and  support  staff  at  the  SBL  indicate  that  the 
ViSSA  system  supported  the  assessment  of  decision-making  skills  to  small  unit  infantry  leaders. 

The  flexible  features  and  functionality  of  ViSSA  support  exercise  monitoring  and  the  statistical 
analysis  work  routinely  performed  at  the  SBL. 

The  improvements  provided  to  the  SBL  by  the  addition  of  ViSSA  are  the  result  of 
developmental  efforts  designed  to  complement  the  software  and  databases  used  at  the  lab  to 
generate  the  3-D  virtual  environment,  not  duplicate  it.  Significant  accomplishments  include  the 
ViSSA  E/C/A  rule  schema  with  additional  user  interfaces  designed  to  define,  sort,  filter,  and  display 
captured  data,  real-time  exercise  monitoring,  and  a  Logger  coupled  with  a  synchronized  Audio 
Logger.  To  ensure  that  the  ViSSA  system  could  achieve  required  performance  levels,  experiments 
were  conducted  at  the  SBL  throughout  development  with  experts  on  staff.  The  Logger/ Audio 
Logger  modules  were  used  to  capture  PDU  traffic  during  execution  of  infantry  training  exercises. 
The  Logger  module  was  invoked  whenever  an  E/C/A  rule  fired  causing  an  annotated  timestamp  to 
be  written  to  a  data  file.  Radio  communications  data  passed  between  the  squad  leader,  the  O/C 
acting  as  platoon  leader,  and  fire  team  leaders  were  also  captured  by  the  Audio  Logger  for 
synchronized  exercise  playback.  The  results  of  the  evaluation  demonstrated  the  system  could  easily 
achieve  the  demanding  performance  requirements  of  multiple  entities  operating  in  the  virtual 
environment.  The  ViSSA  system  is  currently  installed  and  operating  at  three  facilities  within  the 
SBL.  It  is  used  on  a  regular  basis  by  the  SBL  staff  to  log  and  monitor  exercises  and  produce 
statistical  data. 

Utilization  of  Findings: 

The  features  and  functionality  of  ViSSA  have  demonstrated  immediate  application  for  Future 
Combat  Systems  program  research  at  the  SBL.  ViSSA  provides  enhancements  to  the  SBL  software 
configuration  currently  used  at  the  lab,  making  use  of  the  graphical  displays  provided  by  OTB  with 
additional  features  that  apply  meaningful  data  output  and  statistics  to  the  execution  of  simulated 
missions  in  the  virtual  environment.  ViSSA  also  provides  an  efficient,  cost  effective  way  to  monitor 
and  assess  virtual  environment  training  while  greatly  easing  the  burden  on  the  O/C.  These 
capabilities  are  enhancing  the  U.S.  Army’s  ability  to  train  decision-making  skills  to  dismounted, 
small  unit  leaders  (platoon,  squad,  and  team)  in  virtual  environments. 

The  SBL  participates  in  the  validation  of  ideas  for  the  Future  Force  close-fight  battle.  The  force 
structure  and  the  concepts  for  how  the  Army  will  shape  the  close  fight,  and  how  the  Army  will  set 
the  conditions  for  decisive  victory,  are  being  analyzed  and  refined  using  state-of-the-art  modeling 
and  simulation.  Due  to  its  relatively  extensive  use  within  the  SBL,  the  ViSSA  system  has  proven 
capabilities  in  training  and  assessment. 
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DISMOUNTED  INFANTRY  DECISION  SKILLS  ASSESSMENT  IN  THE  VIRTUAL 

TRAINING  ENVIRONMENT 


Introduction 

Dismounted  Soldier  Training  and  Mission  Rehearsal  in  the  Virtual  Environment 

The  military  forces  of  the  United  States  must  increasingly  prepare  for  a  full  spectrum  of 
operations  in  urban  environments  (U.S.  Department  of  the  Army,  2003).  This  significant 
change  from  previous  decades  of  rural  combat  deployments  has  resulted  from  the  emergence  of 
urban  guerrilla  and  state-sponsored  terrorism.  Maintaining  superiority  in  urban  areas  is  often 
key  to  the  control  of  important  resources.  Early  seizure  and  control  of  these  areas  support 
military  objectives  and  successful  follow-on  operations. 

The  emergence  of  advanced  computer  systems  and  computer-generated  models  has  set  the 
stage  for  using  cost-effective,  realistic  simulations  to  complement  urban  operations  training 
facilities  and  better  prepare  the  military  for  combat  in  these  environments  (Sylvester,  2000). 
The  most  advanced  of  these  models  includes  the  ability  to  generate  a  virtual  environment  (VE) 
that  resembles  real  world  terrain  conditions.  The  VE  offers  significant  potential  for  simulation 
options  that  are  transportable  and  can  be  adapted  to  simulate  areas  of  future  operations. 

Figure  1  shows  how  VE  training  can  best  be  utilized  to  enhance  traditional  training 
methods.  Previous  research  (Pleban,  Eakin,  Salter,  &  Matthews,  2001;  Pleban  &  Salvetti, 

2003)  demonstrated  that  VE  training  for  urban  operations  offers  some  significant  advantages. 
These  benefits  are  mainly  in  the  cognitive  areas  of  planning  and  strategy,  decision-making,  and 
team  building.  The  VE  supports  mission  rehearsal  for  the  experienced  Soldier  that  is  otherwise 
very  costly,  involves  safety  issues,  or  requires  frequent  repetition.  Using  a  VE,  Soldiers  can 
experience  a  wide  variety  of  terrains  and  situations  in  a  single  day.  The  training  exercises  can 


Figure  1.  Virtual  environment  augmentation  of  traditional  training  methods. 
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range  from  simple,  single  task  situations  to  complex,  multi-faceted  missions  including  high 
levels  of  (simulated)  danger  and  complex  decision-making  requirements. 

Training  and  Assessing  Small  Unit  Leader  Decision-Making 

The  United  States  military  training  methodology  is  based  on  the  “crawl. .  .walk. .  .run” 
concept  (U.S.  Department  of  the  Army,  1996).  This  concept  of  training  is  especially 
successful  at  teaching  the  critical,  cognitive  decision-making  skills  required  of  Soldiers  and 
small  unit  leaders.  This  methodology  breaks  down  information  into  small,  easily  learned 
segments.  The  tempo,  quantity,  and  complexity  of  the  information  presented  are  increased  as 
the  trainee  demonstrates  understanding  and/or  mastery  of  the  skill  set  being  taught. 

During  preparation  and  planning  for  a  mission,  leadership  and  team  members  typically 
construct  a  baseline  plan  for  operations  and  decision-making.  These  plans  are  composed  of 
combinations  of  pre-structured  task  sequences  they  have  been  trained  to  execute.  These  are 
referred  to  as  tactics,  techniques,  and  procedures  (TTPs). 

Designing  a  mission  plan  based  on  well-practiced  TTPs  allows  commanders  to  make  rapid 
decisions  in  the  face  of  complex  and  dangerous  circumstances.  As  long  as  the  cues  or  patterns 
encountered  by  the  force  support  the  application  of  a  familiar  TI  P,  leaders  coordinate  team 
actions  with  little  need  to  weigh  one  decision  over  another. 

In  contrast,  when  the  circumstances  are  new,  novel,  or  unexpected,  they  require  deliberate 
decision-making.  This  deliberate  decision-making  takes  time  and  can  divert  focus  from  key 
events  and  cues  occurring  in  the  mission  environment.  It  is  impossible  to  duplicate  the 
experience  of  every  possible  situation  that  might  be  encountered  using  virtual  simulation. 
However,  a  representative  set  of  real-world  experiences  can  be  mastered  and  retained  for  future 
mental  reference. 

It  is  difficult,  however,  to  determine  and  assess  the  cognitive  processes  going  on  inside 
someone’s  mind.  The  primary  purpose  of  this  Phase  II  Small  Business  Innovation  Research 
(SBIR)  project  was  the  successful  design  and  deployment  of  a  prototype  system  to  assess  the 
cognitive  decision-making  skills  of  dismounted,  small-unit  (platoon,  squad,  or  team)  leaders  in 
virtual  urban  environments.  The  Virtual  Soldier  Skills  Assessment  (ViSSA)  system  is  the 
operational  prototype  resulting  from  this  effort.  ViSSA  is  capable  of  inferring  decision  making 
based  on  observable  actions  with  sufficient  flexibility  to  handle  the  rapidly  changing  situation 
encountered  in  Military  Operations  on  Urbanized  Terrain  (MOUT). 


Prototype  Software  Requirements,  Design  and  Development 
Development  Process 

ScenPro  applied  the  Scenario-based  Engineering  Process  (SEP)  (Harbison  &  McGraw, 
1997)  to  this  development  effort.  SEP  is  a  methodology  originally  developed  by  the  University 
of  Texas  at  Arlington.  The  Scenario-based  Engineering  Process  is  ideally  suited  to  programs 
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that  have  multiple  users  with  differing  roles;  evolving,  hidden,  or  hard  to  characterize  user 
requirements;  and  information  requirements  that  are  difficult  to  capture.  A  graphic 
representation  of  the  SEP  process  is  show  in  Figure  2.  In  this  development  effort,  we  applied 
all  three  Phases  of  SEP:  Domain  Analysis,  Application  Analysis,  and  System  Implementation. 


Figure  2.  Scenario-based  Engineering  Process. 

Domain  Analysis 

The  Domain  Analysis  phase  of  SEP  focuses  on  analyzing  and  understanding  the  problem 
domain.  This  begins  by  evaluating  the  existing  processes,  technology,  and  tools.  Next, 
Scenarios  of  Use  are  created  with  the  help  of  subject  matter  experts  and  users.  These  scenarios 
cover  both  the  way  things  are  currently  done  in  the  domain  and  how  the  shareholders  envision 
it  could  be  done  in  the  future.  Finally,  the  scenarios  are  analyzed  to  understand  tasks,  data 
flow,  and  constraints. 

In  support  of  Domain  Analysis,  ScenPro  knowledge  engineers  looked  for  existing 
automatic  (virtual  world)  event  detection  systems  and  systems  to  automatically  support  after 
action  reviews.  The  Automated  Training  Analysis  and  Feedback  System’s  (ATAFS)  (Brown, 
Wilkinson,  et  al.,  1997)  goals  were  to  demonstrate  the  capability  to  provide  AAR  leaders  a 
menu  of  automatically  and  manually  generated  aids  for  use  in  the  AAR  within  10  minutes  after 
the  end  of  a  simulation  exercise.  The  ATAFS  was  able  to  meet  this  goal  by  employing 
platforms  with  multi-tasking  capabilities  and  by  using  a  knowledge  database  to  automatically 
generate  AAR  aids  during  exercises.  As  a  result,  the  ATAFS  was  able  to  provide  a  bin  of 
candidate  AAR  aids  available  for  use  at  ENDEX. 

ScenPro  knowledge  engineers  also  spent  time  at  the  Ft.  Benning  and  Ft.  Leonard  Wood  to 
understand  the  Army  Soldier  training  and  assessment  process,  the  Army  leader  training  and 
assessment  process,  the  Army  after  action  review  (AAR)  process,  the  Soldier  Battle  Lab 
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(SBL),  and  the  specific  hardware  and  software  components  within  the  SBL.  The  following 
hardware  and  software  technologies  were  analyzed: 

OneSAF  Objective  System  ( OOS).  The  Program  Executive  Office  of  Simulation,  Training, 
and  Instrumentation  (PEOSTRI)  has  developed  a  set  of  software  modules  and  applications  that 
may  be  used  as  a  baseline  for  constructing  Distributed  Interactive  Simulation  (DIS)  and 
Computer  Generated  Forces  (CGF)  applications.  The  OneSAF  Testbed  Baseline  (OTB) 
applications  were  developed  specifically  to  provide  a  vehicle  for  development,  integration, 
testing,  and  user  feedback  of  technology  developments  for  the  Objective  System.  These  interim 
versions  of  VE  software  are  prototype  steps  in  a  5-7  year  development  plan  by  PEOSTRI  to 
consolidate  and  standardize  the  Army’s  VE  products. 

Dismounted  Infantry  Semi-Automated  Forces  (DISAF)  8.0/9.4.  DISAF  simulates  entities 
for  DIS-based  virtual  environments  that  are  sufficiently  realistic  to  cause  the  illusion  that  the 
displayed  vehicles  and  semi -automated  forces  (SAF)  are  humans,  rather  than  computers.  These 
entities,  which  include  ground  and  air  vehicles,  individual  combatants,  missiles,  and  dynamic 
structures,  can  interact  with  each  other  and  with  manned  individual  combat  simulators  to 
support  training,  combat  development  experiments,  and  test  or  evaluation  studies  on  the  virtual 
battlefield.  DISAF,  like  OTB  1.0,  is  derived  from  Modular  Semi-Automated  Forces 
(ModSAF).  DISAF  enhancements  have  been  migrated  into  the  OTB  2.0  software  distributed 
and  supported  by  PEOSTRI 

Soldier  Visualization  Station™.  The  Soldier  Visualization  Station  (SVS)  is  a  state-of-the- 
art,  high  fidelity,  individual  combatant,  virtual  simulation  system.  It  works  with  DIS-based 
virtual  environments.  It  provides  3-D  graphics,  directional  audio  portals,  and  a  screen 
projection  user  interface  into  the  virtual  environment  (Pleban,  et  al.,  2001).  A  single  Soldier  is 
“instrumented”  by  wearing  a  helmet  with  radio  frequency  sensors  on  it.  He/she  is  equipped 
with  a  radio  and  simulation  rifle  that  is  networked  to  a  control  center  computer.  Detectors  in 
the  room  determine  the  Soldier’s  location  and  posture.  The  Soldier  Battle  Lab  has  more  than 
80  stand-up  SVSs  and  many  more  desktop  SVSs.  There  are  also  desktop  versions  of  the  SVS 
that  allow  the  Soldier  to  use  a  simple  joystick  to  participate  in  the  VE.  This  allows  involving 
more  humans  into  an  exercise  at  a  much  lower  cost  and  is  especially  useful  for  role  players 
when  CGFs  are  inadequate. 

Digital  Audio  Communication  System  (DACS).  Advanced  Simulation  Technology,  inc. 
(ASTi)  developed  the  DACS  for  simulation  and  training  environments.  It  is  a  modular, 
scalable  software  and  hardware  architecture  that  simulates  a  wide  variety  of  radio  networks  and 
conditions.  It  operates  in  DIS  and  High  Level  Architecture  (HLA)  network  environments.  An 
extensive  ASTi  simulated  communications  system  was  installed  at  the  SBL  that  provides  188 
full-fidelity  simulated  radios  and  supports  185  operator  positions  distributed  throughout  three 
warehouse-sized  buildings  covering  an  area  roughly  equivalent  to  on  city  block. 

Dismounted  Infantry  Virtual  After  Action  Review  System  (DIVAARS).  The  Institute  for 
Simulation  and  Training  (1ST)  at  the  University  of  Central  Florida  developed  this  virtual 
simulation  logger  and  playback  system.  DIVAARS  provides  a  real-time  3-D  graphic  display 
(stealth  view)  of  the  exercise  and  during  the  AAR. 
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The  BattleMasters  and  technical  staff  at  SBL  provided  a  comprehensive  understanding  of 
the  virtual  environment  software  and  databases,  network  configuration,  and  expectations  for 
integrating  the  prototype  ViSSA  system.  This  involved  extensive  knowledge  acquisition  and 
observation  of  the  tasks,  events,  interactions,  individual  roles,  and  outcomes  expected  during 
an  assessment  exercise.  Ultimately,  the  end  user  requirements  for  development  of  the  YiSSA 
system  were  defined  and  refined.  Requirements  focused  on  the  needs  of  three  different  user 
groups: 

•  U.S.  Army  trainers,  who  understand  the  skills  required  for  mission  success  as  well 
as  the  process  of  training  and  assessing  Soldiers  and  small  unit  leaders. 

•  The  BattleMasters  and  Observer/Controllers  (O/C),  who  will  interact  with  the  VE 
software  user  interfaces  to  stage  the  assessment  exercise  and  work  with  the  trainer 
to  effectively  capture  and  annotate  decision  points  within  the  scenario  where  the 
Soldier  performed  above  or  below  the  expected  skill  level. 

•  The  trainee,  who  will  be  briefed  about  the  mission  prior  to  entering  the  virtual 
world  and  then  provided  with  a  comprehensive  after-action  review.  The  AAR 
should  include  all  participants  as  a  team-building  exercise,  where  they  may  learn 
from  the  mistakes  of  all  team  members  and  share  together  in  the  victory  of  a 
successful  mission. 

Application  Analysis 

The  Application  Analysis  phase  of  SEP  uses  the  domain  model,  developed  during  Domain 
Analysis,  to  enumerate  requirements  and  design  an  application  that  meets  all  requirements, 
constraints,  and  user  expectations.  This  design  task  can  include  trade-off  studies  and  algorithm 
development  efforts. 

The  key  requirement  of  ViSSA  was  to  automatically  “detect  significant  events”  in  the 
exercise  related  to  small  unit  team  leaders’  cognitive  decision-making  skills. 

In  support  of  the  key  requirement  to  “detect  significant  events,”  the  following  additional 
requirements  were  identified  to  create  a  usable  system: 

•  Minimizing  the  burden  of  the  O/C 

•  Supporting  after-action  reviews  -  which  requires  logging  and  replaying  all  exercise 
Protocol  Data  Unit  (PDU)  packets 

•  Allowing  the  trainer  to  use  legacy  2-D  and  3-D  visualization  to  loop  over  significant 
events  to  aid  in  making  training  points 

•  Capture  and  playback  of  synchronized  audio  (inter-Soldier  radios) 

Additional  exercise  monitoring  functionality  required  by  trainers,  Battlemasters,  and  O/Cs 
includes: 


•  Capturing  and  displaying  real-time  statistics  about  weapons  use,  wounding,  kills,  and 
fratricides 
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Capturing  and  displaying  high-level  status  information  about  SVSs,  radios,  and  SAF 
programs 


Additional  post-exercise  analysis  functionality  required  by  researchers  includes: 

•  Exporting  PDU  data  to  a  database  for  later  analysis 

•  Exporting  statistics  to  a  spreadsheet  for  later  analysis 

•  Exporting  high-level  event  information  to  a  text  file  -  providing  a  high  level  flow  of  the 
events  of  the  exercise 

Trade-off  studies  evaluated  different  architectural  approaches  and  determined  that 
complementing  existing  PEOSTRI  software  and  using  overlay  graphics  to  define  areas  of 
interest  related  to  decision-making  provided  the  most  viable  option  for  capturing  Soldier 
decision-making  within  the  virtual  environment.  This  required  a  careful  review  of  OTB 
(versions  1.0  and  2.0)  and  DISAF  (versions  8.0  and  9.4)  software  and  documentation  produced 
by  PEOSTRI. 

The  key  system  design  considerations  included: 

1 .  Leveraging  existing  technology  and  tools. 

2.  Compatibility  with  the  S VS. 

3.  Compatibility  with  PEOSTRI  OTB  1. 0/2.0  and  DISAF  8.0/9.4  software  and  dynamic 
terrain  database. 

4.  Minimize  the  requirements  placed  on  the  O/C. 

5.  Possibility  of  holding  the  exercise  and  AAR  in  different  locations. 

6.  Ensure  system  can  scale  from  a  single  Soldier  to  thousands  of  Soldiers  and  SAFs  yet  be 
controlled  by  a  single  O/C. 

7.  Assumptions  about  a  Soldier’s  decision-making  skills  must  be  drawn  primarily  from 
his/her  actions.  Specifically,  the  direction  they  choose  to  walk,  what  posture  they  use, 
how  quickly  they  react,  where  they  fire,  and  how  they  direct  their  squad  or  teams  to 
respond. 

8.  The  Soldier’s  squad  may  be  human  beings  assigned  to  a  specific  role  in  the  SVS,  or 
they  might  be  SAF  Soldiers  operating  under  control  of  the  O/C. 

The  final  design  of  ViSSA  resulted  in  three  software  modules:  Logger,  Audio  Logger,  and 
Assessment.  Figure  3  shows  the  ViSSA  Environment  Diagram  indicating  how  the  ViSSA 
components  interact  with  each  other  and  with  other  components  in  the  SBL. 
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Figure  3.  ViSSA  environment  diagram. 


ViSSA  Assessment.  The  Assessment  module  is  used  during  two  distinct  periods.  First,  it  is 
used  in  conjunction  with  the  Assessment  Exercise  Development  Methodology,  well  before  the 
actual  exercise,  to  create  a  scenario  file.  During  this  time  period,  the  exercise  author  works 
with  the  Battlemaster  to  create  both  a  OneSAF/DISAF  scenario  file  (containing  SAFs,  SAF 
movements  and  behaviors,  routes,  overlays,  areas,  lines,  and  points)  and  the  Assessment 
scenario  file  (containing  the  unit  hierarchy,  observations,  and  E/C/ As).  The  Assessment 
Exercise  Development  Methodology  also  calls  for  the  development  of  an  operations  order 
(OPORD)  and  other  mission  briefing  materials.  In  support  of  this  use,  the  Assessment 
module’s  design  includes  dialog  boxes  to  visualize  the  unit  hierarchy  and  the  overlay  graphics 
that  were  created  by  OneSAF/DISAF  -  as  well  as  a  Microsoft  Visual  Basic  Script  (VBScript) 
editor  and  error  checker. 

The  Assessment  module  is  also  used  during  the  exercise.  During  this  period,  the  module  is 
designed  to  be  remotely  controlled  from  the  Logger  module.  The  design  called  for  the  creation 
of  a  ViSSA  COM  Object  Model  to  store  details  of  entities  captured  via  PDU  packets  (IEEE  Std 
1278.  la-1998).  The  module  design  called  for  a  robust  DIS  exercise  scripting  engine  to  support 
arbitrary  assessment  rule  execution.  As  currently  used,  this  feature  allows  for  automating  the 
detection  and  marking  of  significant  events  on  an  exercise  timeline.  The  power  and  flexibility 
of  this  module  allows  it  to  be  used  for  as-yet  unknown  applications  including:  new  PDUs, 
arbitrary  assessments,  adaptive  training  and  assessment,  custom  data  collection,  custom  data 
analysis,  and  custom  data  reporting  (IST-CF-03-01).  Moreover,  using  standard  Microsoft  COM 
Automation™  and  Microsoft’s  VBScript  language  means  that  anyone  who  is  competent  in 
creating  Microsoft  Excel™  macros  can  be  effective  in  creating  custom  automated  solutions. 
The  use  of  Wizards  and  GUI  screens  simplifies  rule/script  entry  for  common  assessment  items. 
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For  situations  where  an  assessment  is  required  for  which  there  are  no  relevant  PDUs,  the 
design  includes  an  Observation  feature  in  which  the  O/C  can  manually  alert  the  Assessment 
module  of  some  specific  Soldier  behavior  that  can  then  be  assessed. 

ViSSA  Logger.  The  key  function  of  the  ViSSA  Logger  is  to  capture  and  record  exercise 
PDU  packets.  The  reasoning  behind  having  a  logger  is  to  enable  visualization  of  Soldier 
actions  (on  existing  2-D  and  3-D  visualization  components),  related  to  a  good  or  poor  decision, 
during  the  after  action  review.  Early  in  the  system  design  phase,  the  design  team  learned  that 
there  were  no  free,  quality  PDU  loggers  available.  The  OTB  Logger,  which  was  originally 
developed  for  ModSAF,  was  unstable  in  the  OTB  environment  (related  to  compression 
software  and  an  updated  entity-state  PDU).  The  MaK  Logger,  which  was  in  use  at  the  SBL, 
was  expensive  and  only  ran  on  Silicon  Graphics  (SGI)  workstations  (which  were  being  phased 
out). 

Users  requested  that  the  Logger  be  user-friendly  and  support  features  such  as  a  graphical 
timeline,  graphical  significant  event  timeline  markers,  pause,  slow/fast  forward  motion, 
slow/fast  reverse  motion,  and  looping.  During  the  AAR,  the  Logger  allows  the  trainer  to 
quickly  focus  on  the  significant  events.  Also,  the  Logger  provides  the  trainer  with  Socratic 
questions  about  the  Soldier’s  actions  to  focus  attention  on  poor,  good,  and  excellent  skills. 

ViSSA  Audio  Logger.  During  a  knowledge  acquisition  trip  to  the  SBL  the  design  team  was 
advised  that  it  was  critical  to  capture  and  replay  synchronous  audio.  With  this  key  feedback, 
the  software  development  team  investigated  the  audio  system  installed  in  the  SBL  -  the  ASTi 
DACS  system.  They  determined  that  the  design  should  include  a  software  module  to  capture, 
store,  and  playback  the  audio. 

The  design  of  the  Audio  Logger  specified  playback  over  the  ViSSA  computer’s  speaker 
before  and  during  the  exercise  as  well  as  playback  over  the  ViSSA  computer’s  speaker  and  the 
DACS  system  after  the  exercise.  A  decision  was  made  to  have  the  Audio  Logger  be  remotely 
controlled  from  the  ViSSA  Logger  -  minimizing  the  burden  placed  on  the  observer/controller. 

System  Implementation 

Implementing  a  system  of  ViSSA’ s  complexity,  under  the  tight  budget  constraints  of  an 
SBIR  program,  was  only  possible  by  using  a  wide  variety  of  support  tools.  ScenPro  chose  to 
implement  ViSSA  using  Microsoft  technology.  The  software  was  written  in  Microsoft  Visual 
Basic  6.  The  system  operates  on  Windows  2000  and  XP  platforms. 

Each  of  the  three  ViSSA  modules  use  WinSock  2.0  to  capture  and  transmit  PDU  packets 
over  the  local  area  network  (LAN). 

The  Audio  Logger  uses  DirectSound  (part  of  DirectX  8a)  to  play  the  DACS  radio 
communications  out  on  the  computer’s  speakers. 
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The  system  is  capable  of  exporting  all  logged  PDU  packets  from  an  exercise  to  Microsoft 
Access™  2000  using  Active  Data  Objects  (ADO)  2.6.  The  system  can  also  export  exercise 
statistics  to  a  Microsoft  Excel  2000  file  using  COM  Automation. 

Using  Microsoft  Scripting  Engine  5.6  and  Script  Control  1.0,  the  Assessment  module 
implements  its  own  Object  Model,  which  can  then  be  operated  on  by  a  scripting  engine.  The 
system  is  currently  set  to  use  VBScript,  although  the  system  can  easily  be  changed  to  support 
other  scripting  languages  such  as  Java  and  Perl.  Additionally,  exercise  developers  can  put 
VBScript  into  the  E/C/A  rules  to  use  COM  Automation  (Steams,  1998).  The  ViSSA  Object 
Model  is  shown  in  Figure  4. 

Various  graphics  objects,  such  as  timeline  markers  and  icons  were  created  using  Microsoft 
Paint  and  Microsoft  Imagedit. 

During  software  development,  Microsoft  Visual  Source  Safe  was  used  for  backup  and 
configuration  management  of  source  and  object  files. 

Microsoft’s  Package  and  Deployment  Wizard  was  used  to  create  the  installation  package. 

The  ViSSA  system  is  designed  to  work  with  any  Compact  Terrain  (v7  -  little  endian) 
terrain  file.  The  majority  of  development  and  testing  was  done  using  the  McKenna  MOUT 
terrain. 
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Figure  4.  ViSSA  object  model. 
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Best  Available  Copy 


The  Virtual  Soldier  Skills  Assessment  System 


The  results  of  this  SBIR  research  and  development  effort  are  an  operational  prototype 
software  system  (ViSSA),  a  structured  methodology  for  developing  assessment  exercises,  a 
network  architecture  for  integrating  ViSSA  into  the  Ft.  Benning  Soldier  Battle  Lab  and  other 
VEs,  and  a  concept  of  operations  for  using  ViSSA  to  assess  cognitive  decision-making  skills. 

ViSSA  System  Components 

The  ViSSA  System  is  composed  of  three  software  modules:  ViSSA  Assessment,  ViSSA 
Logger,  and  ViSSA  Audio  Logger  ( Figure  5).  The  following  sections  describe  the  key  features 
and  use  of  these  modules. 


ViSSA  Assessment  Module  2-D  Plan  View 


Figure  5.  ViSSA  system  component  overview. 

ViSSA  Assessment  module.  The  BattleMaster  uses  the  ViSSA  Assessment  module  to  create 
the  Event/Condition/Action  rules  that  allow  the  O/C  to  monitor  and  assess  Soldier 
performance.  The  ViSSA  Assessment  module  is  a  powerful  tool  in  the  hands  of  a  skilled 
BattleMaster.  In  addition  to  being  able  to  define  arbitrary  E/C/A  rules  using  VBScript  and  the 
ViSSA  Object  Model,  the  user  interface  provides  templates  to  insert  pre-defined  assessment 
rules  that  selectively  monitor  and  capture  specific  events  dining  the  exercise  for  real-time 
analysis. 

ViSSA  operates  by  monitoring  the  PDU  traffic  being  broadcast  (by  SVSs  and  SAF 
programs)  during  a  virtual  environment  exercise.  Using  only  this  PDU  traffic,  ViSSA  creates 
an  internal  model  (Object  Model)  of  the  virtual  environment  including  each  entity's  location, 
orientation,  posture,  equipment,  and  health.  Each  time  an  entity  moves,  fires,  changes  posture, 
makes  radio  contact,  etc.,  their  simulation  sends  out  a  PDU  message  which  is  received  by 
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Assessment.  The  Assessment  module,  in  turn,  raises  an  event  that  can  be  used  to  evaluate  a 
simple  VBScript  rule. 


In  addition  to  the  information  available  in  the  PDUs,  ViSSA  also  supports  the  notion  of  a 
"Phase"  of  the  mission,  an  arbitrary  number  of  timers,  geo-reference  points,  and  user  defined 
variables.  Similarly,  ViSSA  supports  the  ability  for  the  O/C  to  indicate  a  Soldier  behavior  that 
is  not  detectable  via  PDUs  such  as  a  hand  gesture  or  the  content  of  a  radio  message.  These 
“observations”  are  specified  when  the  scenario  is  being  created.  During  the  exercise,  the  O/C, 
who  is  sitting  at  the  Logger,  can  make  an  “observation.”  This  causes  a  special  (ViSSA-only) 
PDU  to  be  sent  from  the  Logger  to  Assessment,  where  it  can  launch  an  E/C/A  rule. 

Tables  1,  2,  and  3  below  show  the  Events,  Conditions,  and  Actions  available  in  the  ViSSA 
Assessment  module. 


Table  1. 

Assessment's  Events 


Event  Name 

Event  Description 

OnStartEx 

Raised  by  the  Logger  when  the  O/C  clicks  the 
“Record”  button  indicating  the  start  of  the  exercise 

OnEndEx 

Raised  by  the  Logger  when  the  O/C  clicks  the 
“Stop  Record”  indicating  the  end  of  the  exercise 

OnTimer 

Raised  by  Assessment  when  a  VBScript-defined 
timer  goes  off 

OnTimerAbort 

Raised  by  Assessment  when  OnEndEx  is  received. 
This  can  be  used  to  process  timers  that  had  not 
completed  by  the  end  of  the  exercise. 

OnMovement 

Raised  by  SVSs  and  OneSAF/DIS AF  programs 
(via  the  Entity  State  PDU)  when  entities  move  or 
otherwise  change  their  state 

OnFire 

Raised  by  SVSs  and  OneSAF/DISAF  programs 
(via  the  Fire  PDU)  when  entities  fire  their 
weapons  (including  grenades,  etc.) 

OnDetonation 

Raised  by  SVSs  and  OneSAF/DISAF  programs 
(via  the  Detonation  PDU)  when  weapons  hit  or 
explode 

OnT  ran  smitChange 

Raised  by  Audio  Logger  when  it  detects  a  Soldier 
toggling  their  radio  (between  Standby  and 

Transmit) 

OnObservation 

Raised  by  Logger  when  O/C  clicks  an  observation 
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Table  2. 

Assessment's  Conditions 

Conditions _ 

Primarily  based  upon  the  VBScript  IF  statement _ 

Main  sources  of  data  are  the  parameters  of  the  Event  and  the  ViSSA 

Object  Model  values _ 

Can  use  the  Properties  feature  to  store  data  for  future  comparison  (such 
as  starting  time,  starting  location,  starting  health,  and  starting 

orientation) _ 

Can  use  system-level  variables:  Phase,  Start  Time,  and  Elapsed  Time 


Table  3. 

Assessment’s  Actions 

Actions _ 

Change  Phase _ 

Store  Value  in  a  Property  (including  a  time,  a  location,  a  callsign,  an 

arbitrary  object,  an  arbitrary  collection  of  objects,  etc.) _ 

Start/End/Reset  a  Timer _ 

Mark  Timeline  with  a  Good  or  Poor  Significant  Event  or  a  Comment 


Using  these  tools,  E/C/A  rules  can  be  developed  to  assess  small  unit  leader  cognitive 
decision-making  skills.  Some  of  the  different  aspects  of  the  decision-making  process  that  can 
be  assessed  using  this  technology  include: 

•  Route  Selection 

•  Distance  Off  Route 

•  Movement  into  and  out  of  Inclusion/Exclusion  Zones 

•  Use  of  Cover  and  Concealment 

•  Effective  Use  of  Communications  (based  on  ViSSA-only  PDU  from  the  Audio 
Logger  indicating  a  particular  Soldier  used  the  radio) 

•  Use  of  Environmental  Cues 

•  Situational  Awareness  (reacting  properly  to  cues) 

•  Response  to  threat 

•  Reaction/Response  Times 

•  Team  Control 

•  Effective  Use  of  Individuals 

•  Safety  and  Security 

•  Weapon  accuracy 

•  Kills  (Enemy,  Neutral,  Civilian,  Friendly) 

•  Meeting  Deadlines 

•  Performance 
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Leader  decision-making  skills  can  also  be  monitored  through  the  actions  of  his/her  team 
(assuming  the  team  is  following  the  direction  of  the  leader).  ViSSA  includes  a  feature  to  assign 
roles  to  each  Soldier.  E/C/A  rules  can  be  authored  to  monitor  activity  by  role.  For  example, 
team  members  with  the  role  of  rifleman  should  be  the  first  to  enter  areas  and  buildings  while 
the  fire  team  leader  should  be  responsible  for  communications  to  the  squad  leader.  Additional 
decisions  that  can  be  inferred  about  teams  include: 

•  Splitting/joining  teams 

•  Flanking/encircling  actions 

•  Exhibiting  a  defensive  posture 

As  a  more  detailed  example,  the  following  rules  can  be  created  regarding  decision-making 
skills  associated  with  a  room  clearing  exercise: 

•  Check  that  the  building  was  cleared 

•  Check  that  at  least  two  Soldiers  entered  each  room  in  the  building 

•  Check  that  the  other  fire  team  maintained  a  perimeter  around  the  building  - 
defending  the  fire  team  inside  the  building 

•  Check  that  the  building  search  begins  within  XXX  seconds  of  the  start  of  the  Phase 

•  Check  that  all  rooms  in  the  building  are  searched  within  XXX  seconds 

•  Check  that  a  Soldier  "paints"  each  wall  in  each  room  (by  monitoring  their 
orientation) 

•  Check  that  Soldiers  stay  in  each  room  for  at  least  XXX  seconds 

•  Check  that  a  member  of  the  squad  radios  the  commander  within  XXX  seconds  after 
the  search  is  completed 

•  Check  that  after  one  Soldier  enters  a  room,  a  second  enters  the  room  within  XXX 
seconds 

The  ViSSA  Assessment  module  has  been  developed  to  quickly  select  basic  Event 
Condition/ Action  rule  templates  or  create  E/C/A  rules  written  in  VBScript.  The  ViSSA 
Assessment  module  detects  the  Events  as  Soldier  movement  and  behavior  PDUs  are 
transferred  over  the  simulation  network  via  DIS/HLA.  For  each  type  of  Event,  such  as  weapon 
fire  or  movement,  there  is  an  event  handler  listed  on  the  left  side  of  the  Assessment  module 
screen,  as  seen  in  Figure  6.  The  VBScript  code  that  describes  the  Condition  and  the  resulting 
Action  is  entered  into  the  corresponding  event  handler  field.  When  an  Event  occurs,  the  code  is 
then  run  to  see  if  the  conditions  are  met  and  which  Action  should  follow.  An  example  is  a 
Friendly  Fire  rule  (Figure  6).  The  Event  would  be  the  firing  of  a  weapon  (which  is  handled  by 
the  OnFire)  event  handler. 
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Figure  6.  The  ViSSA  Assessment  module  user  interface. 

The  VB  Script  is  inserted  into  the  OnFire  event  handler  to  check  the  force  of  the  shooter 
and  target.  In  this  example,  if  both  shooter  and  target  are  Blue  Forces  (BLUFOR),  the  resulting 
Action  is  a  “Poor”  mark  on  the  exercise  timeline  of  the  logger. 

In  the  case  of  the  E/C/A  rule  templates  (Figure  7),  the  code  is  automatically  inserted  into 
the  proper  event  handlers.  E/C/A  rules  provide  the  O/C  with  a  cost-effective  technology  to 
specify  which  Soldier  movements  and  behaviors  demonstrate  mastery  of  the  predetermined 
training  and  learning  objectives.  In  addition,  Soldier  decisions,  actions,  and  behaviors  of  grave 
concern  such  as  fratricide  can  be  preprogrammed  for  immediate  system  alert  to  the  O/C. 


Areas  Exclusion  Rtie  Template 


Rule  Name 
Phase 


Ares  Name 
Entity 


(Stay  Away  from  Windows 


OK 


C  AB  Phases 

Specific  Phase  (Pn^Assauft 

Zones  “  3 


Cancel  j 
Help  [ 


S51A1 
gl  A2 
l*)1A3 


(A L 


S  elect  All  j 
Pear  Al  [ 


AAR  Questions 

[Did  you know you  w«re  at  the  enemy  $  location? 

I  Did  you  rea&20  that  you  were  standing  in  front  of  a  window? 


Figure  7.  Rule  template  dialog  box  (area  exclusion  rule). 
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ViSSA  Logger.  The  ViSSA  Logger  acts  as  the  run-time  user  interface  that  captures  and 
stores  all  PDUs  from  the  virtual  exercise  and  displays  a  time-stamped  summary  of  real-time 
events  (assessments  of  significant  events,  fires,  hits,  kills,  etc).  The  exercise  time-line  allows 
the  BattleMaster  to  make  observations,  manually  add  significant  event  markers,  or  monitor 
automated  significant  event  markers  received  and  recorded  from  the  ViSSA  Assessment 
Module  using  pre-defined  assessment  criteria. 

An  image  of  the  Logger’s  user  interface  is  shown  in  Figure  8.  The  top  includes  VCR-like 
controls  to  record  and  playback  the  exercise.  Along  the  timeline,  green,  up-arrows  indicate 
good  decision-making,  red,  down-arrows  indicate  poor  decision-making  (as  defined  by  the 
exercise  developer),  and  grey,  up-arrows  are  comments.  During  the  exercise,  the  O/C  can 
manually  enter  significant  events.  The  lower  part  of  the  Logger  shows  detailed  information 
about  each  significant  event. 


Following  the  virtual  exercise,  the  ViSSA  Logger  is  an  effective  AAR  tool  providing  a  full 
playback  capability.  In  particular,  the  Logger  will  playback  all  captured  PDUs  over  the  LAN. 
Playing  back  these  PDUs  allows  the  exercise  to  be  re-lived  via  2-D  and  3-D  viewing 
applications  such  as  OneSAF,  DISAF,  and  stealth  viewers.  This  allows  the  BattleMaster,  O/C, 
or  trainer  to  quickly  focus  on  key  decisions  and  events  that  occurred  during  the  virtual 
exercise.  When  there  is  limited  time  for  an  AAR,  the  trainer  can  quickly  illustrate  the  tactical 
situation  that  should  have  elicited  a  response  from  the  trainee,  or  selectively  focus  on  positive 
or  negative  outcomes  of  decision-making  by  the  trainee  (e.g.,  show  how  the  squad  took  direct 
fire  when  the  leader  trainee  commanded  crossing  an  open  area  without  cover).  Other  features 
include  playing  forward/backward,  looping,  playing  at  a  faster  speed  and  in  slow  motion. 
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Figure  8.  ViSSA  Logger  graphical  user  interface. 
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The  Logger  provides  a  number  of  administrative,  analysis,  and  reporting  tools  to  support 
the  BattleMaster  and  O/C  during  the  exercise  and  to  support  the  Trainer  and  Researcher  after 
the  exercise.  During  the  exercise,  the  Events  dialog  box  (Figure  9)  displays  a  brief  (or  verbose) 
summary  of  all  the  key  events  that  occurred  during  the  exercise. 
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00:09:25 
00:09:25 
00:09:25 
00:09:26 
00: 09:26 
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00:09:26 
00: 09: 26 
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3bi*j 
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0P1  Hissed  with  Rounds 

0P1  Fired  at  Unknown  (89.251.1001)  with  Rounds 

0P1  Hissed  with  Rounds 

2A1  Fired  at  0P1  with  Rounds 

2A1  Hissed  with  Rounds 

2A1  Fired  at  0P1  with  Rounds 

2A1  HIT  0P1  with  Rounds 

2A1  Fired  at  0P1  with  Rounds 

2A1  KILLED  Enemy  0P1  with  Gun 

2A1  Hissed  with  Rounds 


J 


3 

r  Lock  Updates 


Figure  9.  Logger  events  dialog  box. 


During  the  exercise,  the  Logger  provides  a  Mission  Status  dialog  box  (Figure  10)  showing 
high  level  statistics  about  each  Soldier.  These  statistics  are  organized  by  callsign  and  include 
their  health  and  details  of  their  weapons  use  (fires,  hits,  kills,  etc.). 


The  PDU  Statistics  dialog  box  (Figure  1 1)  is  provided  by  the  Logger  to  support  the 
BattleMaster  and  O/C.  It  displays  a  brief  summary  of  the  number  of  PDUs  of  different  types 
that  have  been  received  by  the  Logger.  This  is  useful  to  ensure  that  all  of  the  components  of  the 
Soldier  Battle  Laboratory  are  operating  properly. 
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"  PDU  Statistics  *; 


Live  PDUs  Received: 

Tolat  0 
Error*: 

Bad  Protocol  Version:  0 
Unexpected  Exercised  ID:  0 

Recorded  PDUs 

POU  Type  Counts: 

ent^y  state  (It  30848 
fee  (2):  40 
detonation  (3):  40 
edition  (4):  4 
start_resume  (1 3):  3 
acknowledge  (15):  50 
dat$_que»yD8):  210 
set_data  (1 9):  1448 
data  (20):  19423 


Figure  11.  Logger  PDU  statistics  dialog  box. 

The  Logger  allows  the  user  to  export  data  in  three  ways: 

•  The  Event  Summary  data  (Figure  9)  can  be  exported  to  a  simple  text  file 

•  The  Mission  Status  data  (Figure  10)  can  be  exported  to  MS  Excel  (Figure  12) 

•  The  PDU  data  can  be  exported  to  MS  Access  database  (Figure  13). 
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Figure  12.  Logger  statistics  data  exported  to  MS  Excel. 
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Q  Microsoft  Access  -[sample: 
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Figure  13.  Logger  PDU  data  exported  to  MS  Access. 


The  MS  Access  database  that  is  created  by  the  Logger  includes  all  PDU  data  captured 
during  the  exercise  as  well  as  a  large  number  of  enumeration  (ENUM)  tables  used  to  translate 
the  numeric  tags  to  English  descriptions.  The  database  is  pre-loaded  with  19  common  queries 
(Figure  14)  that  can  extract  information  such  as  who  shot  whom. 
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Figure  14.  Example  MS  Access  query:  Show  all  entity  appearance  changes. 
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ViSSA  Audio  Logger.  The  ViSSA  Audio  Logger  captures  and  stores  audio  PDUs  for 
synchronized  audio  playback  in  conjunction  with  the  ViSSA  Logger,  as  shown  in  Figure  15.  It 
allows  for  quick  identification  of  who  is  speaking  on  the  radio,  muting,  and  isolation  of 
specific  radio  channels  by  the  BattleMaster.  Additionally,  it  forwards  communication-related 
data  to  the  ViSSA  Assessment  Module.  This  allows  exercise  developers  to  create  E/C/A  rules 
to  evaluate  Soldier  decision-making  from  the  standpoint  of  communications  (e.g.,  did  the  team 
leader  provide  a  Situation  Report  (SITREP)  to  the  squad  leader  shortly  after  the  sniper  began 
firing). 
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Figure  15.  ViSSA  Audio  Logger  graphical  user  interface. 


The  ViSSA  Audio  Logger  display  is  expandable  (vertically)  based  on  the  number  of  radios 
used  during  the  exercise.  It  provides  a  convenient  way  to  "see"  who  is  talking  during  an 
exercise  and  provides  the  ability  to  add  meaningful  text  descriptions  to  each  radio  channel. 

Assessment  Exercise  Development  Methodology 

Scenario-based  training  provides  a  dynamic  environment  for  Soldiers  to  learn  and 
demonstrate  cognitive  decision-making  skills.  MOUT  is  clearly  the  most  demanding  of  the 
environments  Soldiers  must  operate  in  (Gorman,  2000).  The  VE  shows  promise  in  providing 
Soldiers  the  opportunity  to  prepare  for  MOUT  situations  in  a  safe  and  supportive  training 
environment.  Scenario  experiences  provide  Soldiers  with  a  knowledge  base  to  quickly  draw 
from  when  instantaneous  cognition  and  decisions  are  necessary. 

The  evolution  of  the  VE  now  provides  Soldiers  with  a  broad  selection  of  simulated 
environments  for  training  and  assessment  at  a  single  location.  It  is  however  necessary  to 
structure  the  exercise  so  that  it  resembles  actual  or  plausible  challenges  for  it  to  be  of  value  for 
Soldiers  and  their  organization.  The  scenario  development  process  must  also  be  structured  in  a 
way  to  ensure  that  it  supports  current  and  evolving  doctrine  in  order  to  maintain  exercise 
relevance  and  expansion  of  the  VE  as  a  viable  training  and  assessment  option. 

Soldiers  can  gain  knowledge  and  skills  in  several  ways  including  classroom,  tabletop 
exercises,  field  exercises,  and  deployment  experience.  Once  small  unit  team  leaders  have  a 
minimum  level  of  skill,  they  can  be  assessed  using  ViSSA. 

At  the  highest  level,  the  exercise  development  methodology  is  composed  of  five  phases 
(Table  4).  The  five  phases  are  adaptable  to  VE-based  assessment  either  pre-exercise,  during  the 
exercise,  or  post-exercise.  The  phases  are  detailed  in  the  table  below: 
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Table  4. 


Five  Phases  of  Assessment  Adapted  to  the  VE 


Phase 

Purpose 

Time  Frame 

Assessment 

Exercise 

Development 

Identification  of  the  assessment 
participants,  needs  analysis, 
prerequisites,  equipment,  safety 
issues,  and  facility  selection.  Design 
of  exercise  including  objectives, 
entities,  obstacles,  and  decision¬ 
making  skills  to  assess.  Selection  of 
terrain.  Selection  of  key 
geographical  locations. 

Development  of  briefing  materials. 

Well-prior  to  the  Exercise 

VE 

Familiarization 

Introduce  Soldier  to  virtual 
environment 

Prior  to  the  Exercise 

Pre-Mission 

Briefing 

Presentation  of  the  training/mission 
objectives 

Immediately  Prior  to  the 
Exercise 

Assessment 

Provide  trainee  the  opportunity  to 
exhibit  the  desired  knowledge  or 
skill  during  a  real-world  scenario  in 
the  Virtual  Environment 

During  the  Exercise 

Evaluation 

Conduct  After  Action  Review 
representing  trainee  strengths  and 
weaknesses  observed  during 
demonstration  of  skills 

Post-Exercise 

As  part  of  this  SBIR  effort,  ScenPro  developed  a  methodology  for  creating  assessment 
exercises.  The  ViSSA  Assessment  Exercise  Development  Methodology  (AEDM)  is  a  12  step 
procedure  for  creating  a  VE-based,  decision-making  assessment  scenario. 

The  Assessment  Exercise  Development  Methodology  steps  are: 

1 .  Identify  Soldier  skill(s)  being  assessed 

2.  Review  source  of  Soldier  knowledge 

a.  Training  Material 

b.  Documentation 

3.  Determine  Scenario  Complexity  Level 

a.  Introductory 

b.  Basic 

c.  Intermediate 

d.  Advanced 
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4.  Create  a  scenario,  with  believable  context,  to  stress  one  or  more  of  the  skills.  Document 
using  the  5-paragraph  OPORD  format. 

5.  Determine  expected  Soldier  actions  within  the  scenario  (based  upon  Scenario 
Complexity  Level) 

6.  Indicate  key  decision  points  within  the  scenario  relative  to  the  skills  being  assessed 

7.  Identify  observable  actions  that  can  be  used  to  quantify  the  Soldier’s  application  of  the 
skill  being  assessed 

8.  Determine,  in  conjunction  with  the  O/C,  how  to  capture  each  action 

a.  Directly  Observable  with  PDUs 

1.  Motion  of  Soldier  or  SAP  into  or  out  of  Inclusion  or  Exclusion  Area 

2.  Motion  of  Soldier  or  SAF  across  Phase  Line 

3.  Fires  by  Soldier  or  SAF 

4.  Wound/Kill  of  Soldier,  Teammate,  or  SAF 

5.  Radio  Transmission  by  Soldier  or  O/C 

6.  Exercise  Time  (minutes  since  the  start  of  the  exercise) 

b.  Computable 

1.  Reaction  Time  (time  between  one  action  and  another) 

2.  Facing  (when  two  entities  are  facing  each  other) 

3.  Distance  (between  Soldier  and  target  or  rally  point) 

c.  Inferable 

1 .  Team  Behavior  (over  march,  covering  fire,  coordinated  assault) 

2.  Complex  Routes  (to  avoid  kill  zones) 

3.  Sophisticated  use  of  Weapons  (smoke  and  flash-bangs) 

d.  Observations  by  Observer/Controller 

1 .  Hand  gestures 

2.  Content  of  radio  communications 

9.  Create  a  new  scenario  in  DISAF/OneSAF 

a.  Lay  down  forces  (Friendly,  Neutral,  Enemy) 

b.  Create  overlay  graphics  and  enter  required  points,  lines,  areas,  and  SAF  triggers 
-  annotate  each 

10.  Create  new  ViSSA  scenario  using  Assessment  module 

a.  Specify  which  forces  are  SAF  and  which  should  be  Assessed 

b.  Specify  specific  Observations  the  O/C  can  make  during  exercise 

c.  For  each  E/C/A,  include  Socratic  questions  to  support  AAR 

11.  Create  Mission  Briefing  material  to  give  to  the  Soldier  prior  to  entering  the  SVS 

12.  Create  instructions  for  the  O/C 

As  shown  in  Step  3  above,  the  AEDM  includes  the  concept  of  Scenario  Complexity  Levels 
(SCL)  to  establish  metrics  for  VE  exercises.  The  SCL  provides  a  structure  for  trainers  to  design 
and  execute  scenario  missions  that  meet  the  needs  of  the  individual  Soldier. 

Scenario  Complexity  Level  1:  Introductory.  Introductory  level  scenarios  allow  trainees 
to  get  comfortable  with  the  VE  technology  in  a  low  stress  environment.  Trainees  will 
have  to  manage  minimal  skills  to  accomplish  the  terminal  learning  objective  (TLO). 

See  example  Level  1  scenario  in  Appendix  B. 
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Scenario  Complexity  Level  2:  Basic  Skills.  Level  2  scenarios  provide  an  opportunity  for 
the  trainee  to  manage  a  combination  of  skills  and  is  suited  for  trainees  without 
extensive  experience  or  training  in  the  primary  TLO.  See  example  Level  2  scenario  in 
Appendix  C. 

Scenario  Complexity  Level  3:  Intermediate  Skills.  Level  3  scenarios  provide  an 
opportunity  to  apply  experience  while  managing  several  skill  sets.  Level  3  scenarios  are 
best  suited  for  operationally  experienced  trainees.  See  example  Level  3  scenario  in 
Appendix  D. 

Scenario  Complexity  Level  4:  Advanced  Skills.  Level  4  scenarios  provide  an 
opportunity  for  the  trainee  to  apply  experience  and  skills  in  a  dynamic,  demanding 
environment.  The  trainee  will  be  required  to  manage  multiple  skills  to  meet  the  TLO. 
Level  4  scenarios  are  best  suited  for  trainees  with  both  operational  and  VE  experience. 
See  example  Level  4  scenario  in  Appendix  E. 

As  described  in  Step  9  above,  the  BattleMaster  uses  OneSAF/DISAF  to  pre-define  the 
semi-automated  forces  and  other  features  of  the  scenario  to  establish  the  appropriate  SCL.  In 
particular,  a  scenario  may  include  the  following  items  to  increase  complexity: 

•  Opposing  Forces 

•  External  distractions  to  BLUFOR  activities 

•  Time  constraints  on  BLUFOR  activities 

•  Requirements  to  manage  personnel 

Network  Architecture 

The  ViSSA  system  can  be  configured  in  two  ways.  The  first,  shown  in  Figure  3  above,  is 
appropriate  for  a  facility  where  the  exercises  occur  in  the  same  room  as  the  after-action 
reviews.  The  second,  shown  in  Figure  16  below,  is  appropriate  for  a  facility  that  has  separate 
exercise  and  AAR  areas  and  holds  back-to-back  exercises. 
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Figure  16.  ViSSA  system  component  integration  with  existing  SBL  network  configuration. 


The  ViSSA  System  Concept  of  Operation 

Before  the  exercise.  Prior  to  an  exercise,  a  trainer  or  commander  will  meet  with  the  virtual 
environment  O/C  or  BattleMaster  to  develop  an  exercise  scenario  that  will  assess  a  particular 
Soldier  skill  or  set  of  skills  (e.g.,  the  effective  use  of  cover  and  concealment).  Following  the 
ViSSA  Assessment  Exercise  Development  Methodology,  a  scenario  is  created  and 
Event/Condition/ Action  rules  are  defined. 

Immediately  prior  to  the  assessment  exercise,  Soldiers  are  given  an  opportunity  to  get 
familiar  with  the  VE  and  then  given  a  traditional  mission  brief. 

During  the  exercise.  During  the  exercise,  the  O/C  can  monitor  the  activity  of  the  system 
and  the  trainees  using  the  Events  and  Mission  Status  displays.  The  O/C  can  move  freely 
through  the  simulated  environment  in  a  stealth  mode  using  the  3-D  stealth  view  or  can  monitor 
movement  using  the  2-D  plan  view  display.  The  O/C  can  manually  mark  key  decisions  or 
events  observed,  and  monitor  the  automated  capture  of  key  events  resulting  from  predefined 
O/C  criteria.  The  ViSSA  Assessment  Module  automatically  evaluates  assessment  mles  against 
Soldier  performance  as  the  exercise  proceeds,  looking  for  significant  events  and  conditions  that 
were  defined  before  the  exercise  and  identified  for  capture.  When  the  appropriate 
event/condition(s)  are  met,  the  system  flags  the  exercise  timeline  located  on  the  ViSSA  Logger 
Module  interface  with  a  mark  indicating  a  Good  or  Poor  decision. 
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After  the  exercise.  Once  the  exercise  is  completed,  the  BattleMaster  or  O/C  can 
immediately  begin  the  AAR.  The  ViSSA  Logger  and  ViSSA  Audio  Logger  are  used  to 
playback  logged  PDU  data  with  synchronized  radio  communications  from  the  exercise.  The 
playback  can  be  watched  across  a  DIS  network  using  the  OTB  2-D  Plan  Viewer  and  various  3- 
D  stealth  viewers.  The  ViSSA  Logger  displays  an  exercise  timeline  with  color-coded 
markings  showing  each  of  the  significant  events  from  StartEx  to  EndEx  that  were  manually 
identified  by  the  O/C  or  triggered  automatically  in  accordance  with  the  Event/  Condition/ 
Action  rules.  Selecting  a  mark  on  the  timeline  initiates  playback  of  the  exercise  leading  up  to 
the  significant  event.  The  VCR-like  controls  support  pause,  fast/slow,  reverse,  and  looping. 

This  allows  the  Battlemaster,  O/C,  or  trainer  to  quickly  focus  AAR  activities  on  key  decision 
points  during  the  mission.  The  O/C  also  has  access  to  the  Event  and  Mission  Status  displays. 
The  AAR  discussions  are  facilitated  by  a  set  of  Socratic  questions  about  the  event. 

ViSSA  Prototype  Demonstrations 

During  the  development  effort,  several  opportunities  were  provided  for  observation, 
experimentation,  and  evaluation  at  the  SBL  facility.  These  opportunities  included  participation 
in  Army  Research  Institute  (ARI)  experiments,  technology  demonstrations,  and  ViSSA  system 
testing.  These  opportunities  are  described  in  more  detail  below: 

Mission  rehearsal  experiment  (July  2002).  This  research  examined  the  use  of  virtual 
environments  as  a  viable  dismounted  infantry  mission  rehearsal  tool  (see  Pleban  &  Salvetti, 
2003).  A  prototype  ViSSA  system  was  successfully  used  to  augment  the  AARs  provided  at  the 
SBL.  Numerous  improvements  were  made  to  the  design  of  the  system  based  upon  this 
experience. 

Science  and  Technology  Objective  (STO)  Culminating  event  (September  2002).  The  ARI 
Simulator  Systems  Research  Unit  and  the  University  of  Central  Florida  Institute  for  Simulation 
and  Training  conducted  technology  demonstrations  at  the  SBL  using  DIVAARS.  The  event 
represented  the  culmination  of  a  4-year  STO  that  was  established  to  address  selected 
technological  and  training  issues  related  to  high  fidelity  dismounted  Soldier  simulation.  This 
event  provided  the  ViSSA  development  team  with  an  opportunity  to  observe  a  formal  AAR 
and  the  types  of  summary  statistics  of  value  and  interest  to  the  trainer.  A  great  deal  of  useful 
information  was  learned  about  AAR  tools  during  this  event. 

Joint  Conflict  and  Tactical  Simulation  (JCATS)  4.1  exercise  (February  2004).  ScenPro  was 
asked  to  support  a  large,  classified  exercise  at  the  Ft.  Benning  SBL  that  included  all  available 
SVSs  and  JCATS  4.1.  JCATS  simulates  large  groups  of  Soldiers.  The  SBL  Battlemaster  used 
ViSSA  to:  compute  exercise  statistics,  record  the  exercise,  and  visualize  the  system 
configuration  and  status.  Many  exercises  were  held  during  several  weeks  at  the  SBL.  The 
number  of  entities  participating  in  the  exercises  varied  between  600  and  1900.  Feedback  from 
the  exercise  indicated  that  ViSSA  was  highly  valued  for  it  ability  to  display  configuration  and 
status  information,  record  and  playback  PDUs,  and  to  compute  statistics  in  real  time. 
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Methods 


Overview 

Just  prior  to  the  completion  of  the  contractual  period,  a  capability  assessment  evaluation  of 
the  ViSSA  system  was  performed  at  the  Soldier  Battle  Laboratory  in  Ft.  Benning  by  the 
BattleMasters.  This  evaluation  took  place  in  November  2003.  The  evaluation  included  a 
presentation  about  the  system,  training,  and  an  assessment  exercise.  A  US  Army  Soldier 
participated  in  the  demonstration  by  entering  the  VE  via  an  SVS  and  performing  a  small  task. 
After  the  training  the  Battlemasters  completed  a  pen  and  paper  questionnaire  about  the 
capabilities  of  the  system. 

Preparation 

Successful  installation  and  testing  of  ViSSA  took  approximately  4  hours.  Following 
installation  a  presentation  was  given  to  BattleMasters  and  other  interested  parties  (See 
Appendix  F).  During  the  training  the  BattleMasters  participated  in  the  authoring  of  the  E/C/A 
rules  for  the  demonstration. 

Participants 

The  participants  of  the  capability  assessment  evaluation  consisted  of  the  three  SBL 
BattleMasters.  These  BattleMasters  were  male,  with  ages  ranging  from  30  to  55,  and  had 
between  1  and  5  years  of  experience. 

Instrument 

The  capability  assessment  evaluation  questionnaire  was  designed  for  this  evaluation  and 
contained  a  total  of  30  questions.  The  first  26  questions  had  quantitative  5-point  Likert  scale 
answers  ranging  from  Strongly  Agree  to  Strongly  Disagree.  The  final  4  questions  asked  for 
qualitative  comments,  feedback,  or  evaluation.  The  questions  focused  on  the  usefulness  of 
features  and  functionality  provided  by  various  components  of  the  system. 

Procedure 

Following  the  training,  a  capability  assessment  evaluation  was  given  to  the  BattleMasters 
to  obtain  their  feedback  regarding  the  exercise  monitoring,  AAR  analysis  and  review  support, 
and  AAR  radio  communications  playback  capabilities.  Participants  were  not  given  any  time 
limits  for  completing  the  questionnaire. 

Analysis 

After  the  capability  assessment  evaluation  questionnaires  were  completed,  the  answers 
were  analyzed  using  simple  averages.  One  participant  did  not  answer  questions  18-26.  Because 
of  the  small  sample  size,  results  should  be  viewed  with  caution. 
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Results 


ViSSA  Evaluation  at  the  Soldier  Battle  Lab 

The  results  (Table  5)  showed  that  BattleMasters  agreed  or  strongly  agreed  with  the 
usefulness  of  95%  of  features  offered  by  the  scenario  authoring  component.  In  addition,  they 
agreed  or  strongly  agreed  with  the  usefulness  of  100%  of  features  and  functionality  provided 
by  all  other  system  components.  BattleMasters  were  asked  for  a  list  of  improvements  to  the 
ViSSA  user  interfaces  and  information  display  methods  that  could  greatly  improve  assessment 
effectiveness.  The  recommendations  included  a  drag  and  drop  function  to  reduce  dependence 
on  drop-down  menus  from  the  various  ViSSA  user  interfaces,  a  “sort  data  by  category”  option 
to  isolate  similar  types  of  information  (Kills,  Fires,  Fratricides)  when  reviewing  exercise 
summary  statistics,  and  a  Select  All/DeSelect  All  feature  to  define  which  Soldiers  will  be 
assessed  by  the  E/C/A  rules.  A  copy  of  the  capabilities  assessment  questionnaire,  containing 
the  actual  number  of  responses  per  question,  is  located  in  Appendix  G. 


Table  5. 

ViSSA  Capability  Assessment  Evaluation  Results 


SYSTEM  CAPABILITY 

FEEDBACK 

PERCENTAGE 

Scenario  Authoring 
(comprising  7  questions) 

Strongly  Agree 

62% 

Agree 

33% 

Exercise  Monitoring 
(comprising  4  questions) 

Strongly  Agree 

92% 

Agree 

8% 

AAR  Analysis  &  Review 
(comprising  10  questions) 

Strongly  Agree 

100% 

AAR  Radio 

Communications 
(comprising  5  questions) 

Strongly  Agree 

100% 

Conclusions  and  Recommendations 

The  Virtual  Soldier  Skills  Assessment  system  is  enhancing  the  U.S.  Army’s  ability  to  train 
decision-making  skills  to  dismounted,  small  unit  leaders  (platoon,  squad,  and  team)  in  virtual 
environments.  The  ViSSA  system  is  supporting  research  into  virtual  environment  training, 
assessment,  modeling,  and  simulation  technology  at  the  Soldier  Battle  Lab  in  Fort  Benning, 
Georgia.  The  extensive  features  and  functionality  of  ViSSA  provide  comprehensive  assessment 
of  Soldier  and  small  unit  leader  cognitive  decision-making  skills  dining  training,  assessment, 
and  mission  rehearsal  exercises.  Moreover,  ViSSA  functionality  can  support  research  and 
development  efforts  including  experimentation  into  future  combat  systems  such  as  providing 
remote  sensor  data  using  robots  and  Unmanned  Air  Vehicles. 

ViSSA  complements  PEOSTRI OTB  (versions  1.0  and  2.0),  DISAF  (versions  8.0  and  9.4) 
software,  and  SVS  software,  providing  an  efficient,  cost  effective  way  to  monitor  and  assess 
virtual  environment  training  while  greatly  easing  the  burden  on  the  Observer/  Controller.  It 
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apply  meaningful  data  output  and  statistics  to  the  execution  of  simulated  missions  in  the  virtual 
environment. 

The  automated  data  collection  and  feedback  system  is  capable  of  accurately  assessing  small 
unit  leader  decision-making  skills  based  on  pre-defined  criteria  encoded  as  E/C/A  rules.  It  is 
capable  of  operating  beyond  the  limitations  of  the  VE  by  taking  advantage  of  the  O/C’s  eyes 
and  ears  and  the  annotated  observations  feature.  It  allows  BattleMasters,  O/Cs,  and  trainers  to 
easily  orchestrate  a  effective,  multi-media,  after-action  reviews.  Finally,  following  the  virtual 
assessment  exercise  and  after-action  review,  the  ViSSA  system  provides  optional  storage  of 
large  quantities  of  logged  data,  using  an  Export  Wizard,  which  produces  an  MS  Access 
database  file  containing  all  PDUs.  This  allows  researchers  to  use  MS  Access  to  view  all  fields 
and  to  produce  custom  queries  or  reports  for  further  in-depth  analyses. 

While  the  primary  perspective  of  this  effort  was  the  assessment  of  dismounted  infantry  in 
general  and  small  unit  team  leaders  in  particular,  every  aspect  of  the  system  is  expandable  to 
the  assessment  of  any  virtual  entity  including  manned  and  unmanned  vehicles  including  tanks, 
boats,  ships,  and  aircraft. 

Overall,  while  there  were  limitations  in  the  evaluation  process,  the  system  has  received 
strong  support  from  BattleMasters,  Observer/Controllers,  and  VE  researchers. 

Impact  of  ViSSA  on  After-Action  Reviews 

The  Army  has  adopted  the  after-action  review  process  as  the  primary  means  of  providing 
feedback  after  collective  training  exercises  (Darling  &  Parry,  2000).  The  effectiveness  of  an 
after-action  review  is  dependent  upon  how  well  the  observer/controller  or  trainer  can  visually 
represent  and  organize  data  captured  during  the  exercise  to  guide  interactive  discussions  and 
recommendations  for  performance  improvement  (Morrison  &  Meliza,  1999).  Dismounted 
Infantry  in  a  MOUT  environment  places  greater  demands  on  O/Cs  due  to  the  rapid  pace  of 
action  and  the  obstruction  of  view  because  of  building  structures. 

In  the  virtual  environment,  the  O/C  must  often  play  the  role  of  a  higher  command  by 
monitoring  and  participating  in  the  radio  transmissions  during  the  exercise.  The  O/C  may  also 
be  required  to  perform  a  variety  of  exercise  control  functions  including  control  of  the  semi- 
automated  forces.  These  activities  compete  for  the  O/C’s  attention,  and  interfere  with  the 
primary  goal  of  performance  evaluation.  By  allowing  the  O/C  to  predefine  performance 
evaluation  criteria,  the  ViSSA  system  will  automatically  capture  key  events  related  to  Soldier 
performance  occurring  during  the  virtual  exercise  for  after-action  review.  ViSSA  automates 
after-action  review  preparation  through  the  use  of  expert  system  rules  to  keep  trainer 
requirements  to  a  minimum  and  avoid  distracting  trainers  from  exercise  control  functions.  The 
need  to  develop  tools  and  procedures  that  can  improve  the  quality  of  training  and  assessment 
exercises  and  timeliness  of  feedback  products  for  the  after-action  review  in  DIS  and  HLA 
environments  has  driven  the  present  research. 
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The  ViSSA  system  dramatically  enhances  the  AAR  capability  by  providing  trainees  with 
immediate  video/audio  playback  of  exercise  events,  and  a  timeline  display  depicting  significant 
events  that  were  captured  during  the  exercise.  Summary  statistics  supplement  the  multi-media 
displays  by  calling  out  killer/victim  information,  reaction  times,  and  important  analytical 
details  buried  within  the  flurry  of  simulation  activity.  Together,  this  information  provides  a 
performance  baseline  for  improved  Soldier  decision-making  in  the  future.  In  addition,  the 
statistical  summaries  can  be  used  to  support  continued  research,  development,  and  analysis. 

Recommendations 

In  order  to  fully  assess  a  Soldier's  decision-making  skills,  it  is  necessary  to  go  beyond 
capturing  the  Soldier's  observable  actions  and  understand  the  important  role  of  verbal 
communication  during  mission  execution.  A  logical  extension  of  ViSSA  might  include 
analysis  of  the  types  and  frequency  of  radio  communications  to  provide  valuable  information 
concerning  confidence,  cognitive  decision-making,  and  leadership  skills  (Evans  &  Christ, 
2003).  Types  of  communication  include  requests  for  information,  status  reports,  and 
commands. 

Extending  ViSSA’s  capability  to  record  and  store  pertinent  data  captured  during 
assessment  exercises  would  create  a  useful  corpus  of  training  and  assessment  related  data. 

Over  time,  this  data  could  be  used  to  generate  statistics  about  areas  where  Soldiers  perform  a 
skill  particularly  well  or  particularly  poorly,  about  prerequisites  for  specific  courses,  and  the 
effectiveness  of  one  training  approach  over  another  that  could  improve  training  in  the  future.  A 
long  term  data  storage  and  analysis  module  could  provide  this  functionality. 

The  following  is  a  list  of  possible  future  enhancements  by  module: 

ViSSA  Assessment 

•  Continue  to  raise  level  of  abstraction  for  rules/scripting 

o  Expand  number  of  Rule  Templates  (Wizards) 

o  Expand  number  of  raised  events 

o  Expand  object  model's  objects  and  the  attributes  automatically  tracked 

•  Create  generic  E/C/A  Rule  Template  Wizard  that  constructs  rules  based  on 
primitives 

•  Intervisibility  support  (automatically  calculating  when  one  entity  can  “see”  another) 

•  Built-in  support  for  aggregating  measurements  from  individual  to  unit  level 

•  Enhance  Area  overlay  graphics  to  understand  buildings: 

o  Automatic  comprehension  of  floors 

o  Automatic  comprehension  of  areas  within  buildings 

■  Buildings,  floors,  rooms 

■  Stairs,  doors,  windows 

o  Add  ability  to  automatically  understand  terrain  file  MES  structures 

•  Recognize  and  reason  about  team  formations 

•  Expand  to  customize  handling  of  more  entity  types 

o  Armor,  Aircraft,  Sensors,  UAVs,  UGV,  FCS,  etc 
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•  Stealth  Viewer  support: 

o  Define  list  of  stealth  viewers  to  be  used  during  AAR 
o  Pre-define  list  of  fixed  stealth  teleport  locations  for  use  during  AAR 
o  Associate  a  stealth  teleport  command  with  timeline  event  marker 

•  Add  tool  to  support  creation/visualization  of  Phases  and  Phase  Transitions 

•  Ability  to  Import/Export  VBScript 

•  Expand  to  allow  for  the  processing  of  new,  currently  undefined  PDU  messages 

•  Replace  Microsoft  WinSock  Control  so  that  all  apps  can  be  run  on  one  computer 

•  Addition  of  numeric  scores  associated  with  Good/Poor  Decision  Making 

•  Infer  Soldier's  ability  to  have  heard  an  environmental  sound 

•  Order  SAF  into  action  (inside  OneSAF  and  DISAF) 


ViSSA  Logger 

•  Expand  statistics  collected  and  displayed 

o  Based  on  feedback 
o  Sub-Total  statistics  at  unit  levels 

•  Allow  user  to  customize  Mission  Summary  columns  displayed  (subset  of  options) 

•  Allow  user  to  customize  Statistics  exported  to  MS-Excel  (subset  of  options) 

•  Allow  user  to  customize  Messages  displayed  (verbosity  level  and  subset  of  options) 

•  Import/Export  of  DISAF/OTB  logger  files 

•  Create  library  of  sample  MS  Access  reports  based  on  PDU  export  to  MS  Access 

•  Stealth  Viewer  support: 

o  Remote  control  of  Stealth  Viewer  during  AAR 
o  Associate  a  stealth  teleport  command  with  a  timeline  event  marker 

■  Teleport  to  fixed  location 

■  Teleport  to  an  entities'  view 

•  Dynamically  change  the  timescale 

•  Tool  to  recover  crashed  log  files 

•  Replace  Microsoft  WinSock  Control  so  that  all  apps  can  be  run  on  one  computer 

ViSSA  Audio  Logger 

•  Add  speech  recognition 

•  Replace  Microsoft  WinSock  Control  so  that  all  apps  can  be  run  on  one  computer 


Multi-Exercise  Support 

•  Support  library  of  ViSSA  Scenarios  (cataloging) 

•  Support  database  of  Exercise  Results  (searching/analysis/reporting) 


Other 

•  Add  small,  DIS-aware,  packet-sniffing  utility 
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Contact  Information 


If  you  are  interested  in  acquiring  a  copy  of  ViSSA,  please  contact  Michael  Gately  of 
ScenPro  at  (972)  437-5001. 
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Appendix  A 
Acronym  List 


AAR 

After- Action  Review 

AEDM 

Assessment  Exercise  Development  Methodology 

ARI 

Army  Research  Institute 

ASTi 

Advanced  Simulation  Technology,  inc. 

BLUFOR 

Blue  Forces 

CGF 

Computer  Generated  Forces 

DACS 

Digital  Audio  Communications  System 

DIS 

Distributed  Interactive  Simulation 

DISAF 

Dismounted  Infantry  Semi-Automated  Forces 

DIYAARS 

Dismounted  Infantry  Virtual  After  Action  Review  System 

ENUM 

Enumeration 

HLA 

High  Level  Architecture 

1ST 

Institute  for  Simulation  and  Training 

JCATS 

Joint  Conflict  and  Tactical  Simulation 

LAN 

Local  Area  Network 

ModSAF 

Modular  Semi- Automated  Forces 

MOUT 

Military  Operations  in  Urban  Terrain 

O/C 

Observer/Controller 

OneSAF 

One  Semi-Automated  Forces 

OOS 

OneSAF  Objective  System 

OPFOR 

Opposing  Forces 

OPORD 

Operations  Order 

OTB 

OneSAF  Testbed  Baseline 

PEOSTRI 

Program  Executive  Office  of  Simulation,  Training,  and  Instrumentation 

PDU 

Protocol  Data  Unit 

SAF 

Semi-Automated  Forces 

SBIR 

Small  Business  Innovative  Research 

SBL 

Soldier  Battle  Lab 

SCL 

Scenario  Complexity  Level 

SEP 

Scenario-based  Engineering  Process 

SGI 

Silicon  Graphics,  Inc. 

SITREP 

Situation  Report 

SME 

Subject  Matter  Expert 

STO 

Science  and  Technology  Objective 

svs 

Soldier  Visualization  Station 

TLO 

Terminal  Learning  Objective 

TTP 

Tactics,  Techniques,  and  Procedures 

VBScript 

Visual  Basic  Script 

VE 

Virtual  Environment 

ViSSA 

Virtual  Soldier  Skills  Assessment 
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Level  1  Training  Mission 

Task  Patrol1 


Conditions 

The  Fire  Team  will  perform  a  scheduled  patrol  as  part  of  a  Peace  Keeping 
Coalition.  The  Fire  Team  is  operating  separately  from  its  squad  and  platoon. 


Task  Standards 

The  Fire  Team  must  patrol  a  geographically  defined  sector  of  urban  terrain.  The 
squad  must  maintain  security  throughout  the  mission.  Casualties  are  not 
acceptable. 


Mission  Features 

■  Requires  the  management  of  multiple  personnel 


Operations  Order 

Situation 

Potential  aggressors  are  categorized  as  armed  and  unorganized  criminal 
elements  whose  objectives  are  to  loot,  destroy  property,  and  terrorize  local 
populace. 

Fire  Team  Delta  will  act  separately  from  the  squad  and  platoon  and  not  possess 
any  support.  The  squad  will  be  dismounted. 

Mission 

Fire  Team  Delta  will  conduct  a  scheduled  routine  patrol  of  a  designated 
geographical  area. 


1  U.S.  Department  of  the  Army  (September  1 994).  Mission  training  plan  for  the  infantry  rifle  platoon  and 
squad  (ARTEP  7-8-MTP).  Washington,  DC:  Author. 
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Execution 


Fire  Team  Delta  will  act  with  good  judgment  in  all  facets  of  day-to-day 
operations.  The  Rules-of-Engagement  (ROE)  are  restrictive.  Soldiers  will  use 
directed  force  only  when  necessary  to  defend  themselves  or  another  from 
serious  bodily  injury  or  death.  No  unwarranted  damage  to  the  structures  or 
disruption  of  the  local  services  will  be  tolerated. 

Fire  Team  Delta  will  patrol  the  southeast  sector  of  the  simulated  city.  The  Team 
will  ensure  that  the  thoroughfares  are  clear  of  non-coalition  forces  and  that  the 
coalition-mandated  curfew  is  enforced.  The  Fire  Team  will  begin  operations  at 
building  C6  and  patrol  once  around  each  of  the  buildings  southeast  of  Rumble 
Road  and  North  Highway.  Fire  Team  Delta  will  conclude  operations  by  returning 
to  building  C6  and  providing  a  SITREP  to  the  Platoon  Leader. 

The  Team  will  effectively  maintain  360-degree  security  throughout  the  mission. 

(The  Team  Leader  will  stipulate  individual  assignments  and  tactics) 

Service  Support 

Soldiers  will  have  issued  equipment  and  weapons. 

Command  &  Signal 

The  Team  Leader  is  in  command  of  the  mission.  The  Team  Leader  will  make  all 
reports  and  requests  to  the  Platoon  Leader.  A  SITREP  is  expected  upon 
observing  any  activity  in  the  assigned  Area  of  Operations  and  upon  completion  of 
the  mission. 

The  primary  call  signs  will  be: 

Papa  Lima-  Platoon  Leader 

Delta  Tango-  Delta  Team  Leader 

(Members  of  the  Fire  Team  will  be  issued  call  signs  at  the  discretion  of  the  Team 
Leader) 
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Training  Guidelines 
General  Instructions 
COALITION  FORCES: 

The  Battle  Master  will  assume  the  role  of  the  Platoon  Leader  for  reporting 
purposes.  The  trainees  will  assume  roles  as  members  of  Fire  Team  Delta.  The 
trainees  will  enter  into  the  simulation  southeast  of  the  city  (Starting  Point).  The 
simulation  will  conclude  when  all  of  the  trainees  arrive  at  the  Rally  Point  (Ending 
Point).  Other  Semi  Automated  Forces  may  be  included  into  the  simulation  as 
additional  Coalition  Forces  engaging  in  patrol  operations.  The  trainee's 
efforts/adherence  to  the  Task  Steps  and  Performance  Measures  will  be 
documented.  The  criteria  for  measuring  the  trainees'  performance  are 
established  in  FM  7-8,  Mission  Training  Plan  for  the  Infantry  Rifle  Platoon  and 
Squad. 


Task  Steps  and  Performance  Measures 

•  Team  Leader  supervises  preparation  of  the  mission, 
o  Review  OPORD 

o  Relay  OPORD  to  Team  members 
o  Distribute  assignments/positions/succession  of  leadership 
o  Plan  and  establish  tactics  to  be  used 
o  Lead  subordinate  trainees 

•  The  squad  moves  to  contact. 

o  Maintains  geographical  orientation 
o  Maintains  security 

Measured  Decision  Points 

This  scenario  includes  2  measured  decision  points  for  the  trainees  to  address. 
The  standard  for  performance  is  GO  or  NO  GO. 

1 .  General  Orientation-  the  trainees  should  move  in  an  easterly  direction 
towards  the  objective  in  the  simulation. 

2.  Maintain  Security-  the  trainees  should  maintain  security  throughout  the 
simulation.  The  trainees  will  not  have  maintained  appropriate  security 
measures  if  at  any  time  an  enemy  Soldier  approaches  within  5  meters  or 
less  to  the  squad. 
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JiftSK. 


Training  Aid  1 


Figure  B-2.  Terrain  map  for  level  1  training  mission 
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Level  2  Training  Mission 

Task  Combat  Patrol2 


Conditions 

The  Fire  Team  will  perform  a  combat  patrol  as  part  of  a  Coalition  Force  to 
support  and  uphold  United  Nation  interests.  The  Fire  Team  is  operating 
separately  from  its  squad  and  platoon. 


Task  Standards 

The  Fire  Team  must  patrol  a  geographically  defined  sector  of  urban  terrain  and 
eliminate  armed  OPFOR.  The  squad  must  effectively  utilize  cover  and  maintain 
security  throughout  the  mission.  Casualties  are  not  acceptable. 


Mission  Features 

■  Requires  the  management  of  multiple  personnel 

■  Requires  the  Use  of  Force 


Operations  Order 

Situation 

OPFOR  is  categorized  as  armed  and  organized  military  element  whose 
objectives  are  to  undermine  UN  objectives  and  disrupt  Coalition  Force’s  efforts. 
OPFOR  typically  operate  independently  of  command  elements  in  small  3  to  7 
member  units.  OPFOR  units  utilize  guerilla  and  insurgent  tactics  and  do  not 
require  or  possess  support  elements  (i.e.  air,  artillery  support,  etc.)  or  equipment 
(i.e.  communications). 

Fire  Team  Bravo  will  act  separately  from  the  squad  and  platoon  and  not  possess 
any  support.  The  squad  will  be  dismounted. 

Mission 


2  U.S.  Department  of  the  Army  (September  1994).  Mission  training  plan  for  the  infantry  rifle  platoon  and 
squad  (ARTEP  7-8-MTP).  Washington,  DC:  Author. 
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Fire  Team  Bravo  will  conduct  a  combat  patrol  of  a  designated  geographical  area 
of  the  city. 

Execution 

Fire  Team  Bravo  will  act  with  good  judgment  in  all  facets  of  day-to-day 
operations.  The  Rules-of-Engagement  (ROE)  are  restrictive.  Soldiers  will  use 
directed  force  against  armed  OPFOR  or  when  necessary  to  defend  themselves 
or  another  from  serious  bodily  injury  or  death.  No  unwarranted  damage  to  the 
structures  or  disruption  of  the  local  services  will  be  tolerated. 

Fire  Team  Bravo  will  patrol  the  southwest  sector  of  the  simulated  city  in  grid  W 
67,  82.  The  Team  will  search  for  and  destroy  armed  OPFOR  units.  The  Fire 
Team  will  begin  operations  at  building  C2,  adjacent  to  North  Highway,  and  patrol 
in  a  northern  direction.  Last  intelligence  received  reports  that  the  grid  should  be 
void  of  non-combatants.  Fire  Team  Bravo  will  conclude  operations  upon  rallying 
with  Alpha  Team  just  north  of  building  A2. 

The  Team  will  effectively  maintain  360-degree  security  throughout  the  mission. 
(The  Team  Leader  will  stipulate  individual  assignments  and  tactics) 

Service  Support 

Soldiers  will  have  issued  equipment  and  weapons. 

Command  &  Signal 

The  Team  Leader  is  in  command  of  the  mission.  The  Team  Leader  will  make  all 
reports  and  requests  to  the  Platoon  Leader.  A  SITREP  is  expected  upon 
encountering  OPFOR  activity  upon  arrival  at  the  rally  point. 

The  primary  call  signs  will  be: 

Papa  Lima-  Platoon  Leader 

Bravo  Tango-  Bravo  Team  Leader 

(Members  of  the  Fire  Team  will  be  issued  call  signs  at  the  discretion  of  the  Team 
Leader) 
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Training  Guidelines 

General  Instructions 

COALITION  FORCES: 

The  BattleMaster  will  assume  the  role  of  the  Platoon  Leader  for  reporting 
purposes.  The  trainees  will  assume  roles  as  members  of  Fire  Team  Bravo.  The 
trainees  will  enter  into  the  simulation  south  of  building  C2  (Starting  Point).  The 
simulation  will  conclude  when  all  of  the  trainees  arrive  at  the  Rally  Point  (Ending 
Point).  The  trainee's  efforts/adherence  to  the  Task  Steps  and  Performance 
Measures  will  be  documented.  The  criteria  for  measuring  the  trainees' 
performance  are  established  in  FM  7-8,  Mission  Training  Plan  for  the  Infantry 
Rifle  Platoon  and  Squad. 

OPPOSING  FORCES: 

Between  3  and  7  OPFOR  (Semi-automated  Forces  or  additional  personnel)  shall 
assume  an  aggressive  role  and  attempt  to  eliminate  Fire  Team  Bravo.  OPFOR 
should  be  grouped  and  act  in  two  separate  elements.  One  element  should  be 
stationed  north  of  building  C3  and  the  other  should  patrol  Rumble  Rd.  OPFOR 
elements  will  not  actively  seek  Fire  Team  Bravo  but  will  agress  them  upon 
contact. 


Figure  C-1.  Mission  layout:  Level  2  training  scenario. 
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Task  Steps  and  Performance  Measures 

■  Team  Leader  supervises  preparation  of  the  mission, 
o  Review  OPORD 

o  Relay  OPORD  to  Team  members 
o  Distribute  assignments/positions/succession  of  leadership 
o  Plan  and  establish  tactics  to  be  used 
o  Lead  subordinate  trainees 

■  The  squad  moves  to  contact. 

o  Maintains  geographical  orientation 
o  Maintains  security 

o  Locates  the  enemy  without  being  detected 
o  Effectively  utilizes  cover  and  concealment 

■  The  squad  reacts  to  contact 
o  Engages  the  enemy 

o  Fixes  the  enemy  to  a  position 
o  Maneuvers  against  the  enemy 
o  Destroys  the  enemy 

Measured  Decision  Points 

This  scenario  includes  4  measured  decision  points  for  the  trainees  to  address. 
The  standard  for  performance  is  GO  or  NO  GO. 

1 .  General  Orientation-  the  trainees  should  move  in  a  northerly  direction 
towards  the  objective  in  the  simulation. 

2.  Maintain  Security-  the  trainees  should  maintain  security  throughout  the 
simulation.  The  trainees  will  not  have  maintained  appropriate  security 
measures  if  at  any  time  an  enemy  Soldier  approaches  within  5  meters  or 
less  to  the  squad. 

3.  Use  of  Cover-  the  trainees  should  maintain  the  use  of  cover  throughout 
the  simulation.  No  more  than  50%  of  the  trainees  in  the  simulation  should 
be  without  cover  at  any  one  time. 

4.  Takes  Action  Upon  Contact-  upon  contact  the  trainees  will  eliminate  the 
enemy  Soldiers.  The  trainees  will  locate  and  maneuver  against  the  enemy 
until  they  are  eliminated. 
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Training  Aid  1 


Mission  Map  of  Terrain 
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Appendix  D 


Level  3  Training  Mission 

Task  Perform  Movement  to  Contact3 


Conditions 

A  squad  is  operating  separately  from  its  platoon  in  a  wartime  environment. 
Intelligence  has  just  been  received  that  a  small  cell  of  enemy  forces  is  preparing 
to  attack  a  column  of  coalition  forces. 


Task  Standards 

The  squad  must  intercept  and  destroy  the  enemy  forces  before  they  attack  the 
column  of  Coalition  Forces.  The  squad  must  maintain  security  throughout  the 
mission.  Casualties  are  not  acceptable. 


Mission  Features 

•  Requires  the  use  of  deadly  force 

■  Requires  the  management  of  multiple  personnel 

■  Time  constraints  exist 


Operations  Order 

Situation 

The  enemy  cell  is  comprised  of  approximately  six  personnel  and  is  equipped  with 
light  arms.  The  cell's  intent  is  to  intercept  a  column  of  Coalition  Forces  passing 
through  an  urbanized  area.  The  cell  will  be  traveling  northwest  from  the  southern 
boundaries  of  the  city. 

Alpha  squad  will  act  separately  from  the  platoon  and  not  possess  any  support. 
The  squad  will  be  dismounted. 

Mission 

Alpha  squad  will  intercept  and  destroy  the  enemy  cell  before  the  cell  has  an 
opportunity  to  attack  the  column. 


3  U.S.  Department  of  the  Army  (September  1994).  Mission  training  plan  for  the  infantry  rifle  platoon  and 
squad  (ARTEP  7-8-MTP).  Washington,  DC:  Author. 
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Execution 


Alpha  Squad  will  act  with  speed,  stealth  and  good  judgment  in  all  facets  of 
operations.  The  Rules-of-Engagement  (ROE)  are  restrictive  in  the  sense  that  no 
unwarranted  damage  to  the  structures  or  disruption  of  the  local  services  will  be 
tolerated. 

The  column  of  Coalition  Forces  will  be  traveling  through  the  urbanized  area  (city) 
at  approximately  0730  hours.  Alpha  squad  will  enter  the  city  at  0645  hours  from 
the  west  and  travel  east.  The  Line  of  Departure  (LD)  will  be  east  of  building  Cl. 
The  squad  must  negotiate  the  abandoned  structures  and  locate  and  destroy  the 
enemy  cell  by  0715  hours.  The  squad  will  then  rally  at  the  intersection  of  Rumble 
Rd.  and  Eagle  Dr.  to  be  picked  up  by  a  troop  transport  vehicle  in  the  passing 
column. 

The  squad  will  effectively  utilize  the  cover  and  concealment  supplied  by  the 
structures  and  maintain  360-degree  security  throughout  the  mission. 

(The  Squad  Leader  will  stipulate  individual  assignments  and  tactics) 

Service  Support 

Soldiers  will  have  issued  equipment  and  weapons.  The  Casualty  Collection  Point 
(CCP)  and  Enemy  Prisoners  of  War  (EPW)  collection  point  is  located  at  the 
intersection  of  Rumble  Rd.  and  Eagle  Dr.  No  supporting  units  are  available. 

Command  &  Signal 

The  Squad  Leader  is  in  command  of  the  mission.  The  Squad  Leader  will  make  all 
reports  and  requests  to  the  Platoon  Leader.  A  SITREP  is  expected  after  crossing 
the  LD  and  upon  completion  of  the  mission. 

The  primary  call  signs  will  be: 

Papa  Lima-  Platoon  Leader 

Alpha  Sierra  Lima-  Alpha  Squad  Leader 

(Members  of  the  squad  will  be  issued  call  signs  at  the  discretion  of  the  Squad 
Leader) 


D-2 


Training  Guidelines 

General  Instructions 

COALITION  FORCES: 

The  Battle  Master  will  assume  the  role  of  the  Platoon  Leader  for  reporting 
purposes.  The  trainees  will  assume  roles  as  members  of  Alpha  Squad.  The 
trainees  will  enter  into  the  simulation  west  of  the  city  (Starting  Point).  The 
simulation  will  conclude  when  surviving  trainees  arrive  at  the  Rally  Point  (Ending 
Point).  The  trainee's  efforts/adherence  to  the  Task  Steps  and  Performance 
Measures  will  be  documented.  The  criteria  for  measuring  the  trainees' 
performance  are  established  in  FM  7-8,  Mission  Training  Plan  for  the  Infantry 
Rifle  Platoon  and  Squad. 

OPPOSING  FORCES: 

Enemy  forces  will  be  situated  in  a  loose  wedge  formation  just  south  of  building 
B5.  The  enemy  will  possess  small  arms.  The  enemy  will  shoot  at  the  squad  upon 
seeing  them.  The  enemy's  formation  will  begin  to  fan  out  if  not  destroyed  within 
10  minutes  from  the  first  weapons  discharge.  An  enemy  Soldier  should  appear 
and  attempt  to  approach  the  Coalition  Forces  from  the  rear  while  they  are 
actively  engaged  with  the  enemy  in  force. 
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Figure  D-1.  Mission  layout:  Level  3  training  scenario. 


Task  Steps  and  Performance  Measures 

■  Squad  Leader  supervises  preparation  of  the  mission, 
o  Review  OPORD 

o  Relay  OPORD  to  squad  members 
o  Distribute  assignments/positions/  succession  of  leadership 
o  Plan  and  establish  tactics  to  be  used 
o  Lead  subordinate  trainees 

■  The  squad  moves  to  contact. 

o  Maintains  geographical  orientation 
o  Utilizes  cover  and  concealment 
o  Maintains  security 

o  Locates  the  enemy  without  being  detected 

■  The  squad  reacts  to  contact 
o  Engages  the  enemy 

o  Fixes  the  enemy  to  a  position 
o  Maneuvers  against  the  enemy 
o  Destroys  the  enemy 

Measured  Decision  Points 
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This  scenario  includes  4  measured  decision  points  for  the  trainees  to  address. 

The  standard  for  performance  is  GO  or  NO  GO 

1 .  General  Orientation-  the  trainees  should  move  in  an  easterly  direction 
towards  the  objective  in  the  simulation. 

2.  Use  of  Cover-  the  trainees  should  maintain  the  use  of  cover  throughout 
the  simulation.  No  more  than  50%  of  the  trainees  in  the  simulation  should 
be  without  cover  at  any  one  time. 

3.  Maintain  Security-  the  trainees  should  maintain  security  throughout  the 
simulation.  The  trainees  will  not  have  maintained  appropriate  security 
measures  if  at  any  time  an  enemy  Soldier  approaches  within  5  meters  or 
less  to  the  squad. 

4.  Takes  Action  Upon  Contact-  upon  contact  the  trainees  will  eliminate  the 
enemy  Soldiers.  The  trainees  will  locate  and  maneuver  against  the  enemy 
until  they  are  eliminated. 
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Training  Aid  1 


Appendix  E 


Level  4  Training  Mission 

Task  Movement  to  Contact4 


Conditions 

The  Squad  will  perform  Movement  to  Contact  during  MOOTW. 


Task  Standards 

The  Squad  must  go  to  a  geographically  defined  sector  of  urban  terrain  and 
eliminate  armed  OPFOR  who  are  attacking  Friendly  Forces.  The  Squad  must 
effectively  utilize  cover  and  maintain  security  throughout  the  mission.  Casualties 
are  not  acceptable. 


Mission  Features 

■  Requires  the  management  of  multiple  personnel 

■  Requires  the  Use  of  Force 

■  Time  constraints  exist 

■  Non-Combatants  are  present 


Operations  Order 

Situation 

OPFOR  is  categorized  as  armed  and  organized  military  element  whose 
objectives  are  to  undermine  UN  objectives  and  disrupt  Coalition  Force’s  efforts. 
OPFOR  typically  operate  independently  of  command  elements  in  small  3  to  7 
member  units.  OPFOR  units  utilize  guerilla  and  insurgent  tactics  and  do  not 
require  or  possess  support  elements  (i.e.  air,  artillery  support,  etc.)  or  equipment 
(i.e.  communications). 

Alpha  Squad  will  act  separately  from  the  platoon  and  not  possess  any  support. 
The  squad  will  be  dismounted. 

Mission 


4  U.S.  Department  of  the  Army  (September  1994).  Mission  training  plan  for  the  infantry  rifle  platoon  and 
squad  (ARTEP  7-8-MTP).  Washington,  DC:  Author. 
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Alpha  Squad  will  hastily  move  to  support  Friendly  Forces  under  attack  at  a  Food 
Distribution  site. 

Execution 

Alpha  Squad  will  act  with  good  judgment  in  all  facets  of  day-to-day  operations. 

The  Rules-of-Engagement  (ROE)  are  restrictive.  Soldiers  will  use  directed  force 
against  armed  OPFOR  or  when  necessary  to  defend  themselves  or  another  from 
serious  bodily  injury  or  death.  No  unwarranted  damage  to  the  structures  or 
disruption  of  the  local  services  will  be  tolerated. 

Alpha  Squad  will  move  to  the  UN  Food  Distribution  Site  located  on  the  ground 
floor  of  building  C4D.The  Site  is  manned  and  supported  by  US  and  Coalition 
Forces,  namely  Echo  Squad  from  your  platoon.  Alpha  Squad  will  provide  support 
to  Charlie  Squad  who  is  currently  under  attack  and  “fixed”  by  OPFOR.  Alpha 
Squad  will  begin  movement  in  two  minutes  from  the  Rally  Point  between  building 
groups  A1  and  A2  on  Rumble  Rd. 

(The  Squad  Leader  will  stipulate  individual  assignments  and  tactics) 

Service  Support 

Soldiers  will  have  issued  equipment  and  weapons.  The  Casualty  Collection  Point 
(CCP)  and  Enemy  Prisoners  of  War  (EPW)  collection  point  is  located  at  the 
intersection  of  Rumble  Rd.  and  Eagle  Dr.  No  supporting  units  are  available. 

Command  &  Signal 

The  Squad  Leader  is  in  command  of  the  mission.  The  Squad  Leader  will  make  all 
reports  and  requests  to  the  Platoon  Leader.  A  SITREP  is  expected  upon  arrival 
at  the  Food  Distribution  Site,  encountering  OPFOR  activity  and  elimination  of 
OPFOR. 

The  primary  call  signs  will  be: 

Papa  Lima-  Platoon  Leader 

Alpha  Lima-  Alpha  Squad  Leader 

(Members  of  the  Fire  Team  will  be  issued  call  signs  at  the  discretion  of  the 
Squad  Leader) 
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Training  Guidelines 

General  Instructions 
COALITION  FORCES: 

The  Battle  Master  will  assume  the  role  of  the  Platoon  Leader  for  reporting 
purposes.  The  trainees  will  assume  roles  as  members  of  Alpha  Squad.  Additional 
personnel  or  Semi-Automated  Forces  will  assume  the  role  of  Echo  Squad  inside 
Building  C4D.  The  trainees  will  enter  into  the  simulation  between  building  A1  and 
A2  on  Rumble  Rd.  (Starting  Point).  The  simulation  will  conclude  when  all  OPFOR 
are  eliminated.  The  trainee's  efforts/adherence  to  the  Task  Steps  and 
Performance  Measures  will  be  documented.  The  criteria  for  measuring  the 
trainees’  performance  are  established  in  FM  7-8,  Mission  Training  Plan  for  the 
Infantry  Rifle  Platoon  and  Squad. 

PPOSSING  FORCES: 

Between  5  and  10  OPFOR  (Semi-automated  Forces  or  additional  personnel) 
shall  assume  an  aggressive  role  and  continually  fire  on  the  occupants  of  building 
C4D,  the  Food  Distribution  Site.  OPFOR  should  be  grouped  and  act  in  two 
separate  elements.  One  element  should  be  stationed  north  of  building  C5  and  the 
other  should  should  be  southeast  of  building  B1 . 


Figure  E-1.  Mission  layout:  Level  4  training  scenario. 


NON-COMBATANTS 

Between  5  and  15  Non-Combatants  should  be  moving  freely  in  and  around 
building  C4. 

Task  Steps  and  Performance  Measures 

«  Team  Leader  supervises  preparation  and  execution  of  mission, 
o  Review  OPORD 
o  Relay  OPORD  to  Team  members 
o  Distribute  assignments/positions/  succession  of  leadership 
o  Plan  and  establish  tactics  to  be  used 
o  Lead  subordinate  trainees 
o  Assigns  each  Soldier  a  target  or  area  to  cover 

■  The  squad  moves  to  contact. 

o  Maintains  geographical  orientation 

o  Maintains  security 

o  Approaches  from  enemy  flank  or  rear. 

o  Moves  rapidly 

o  Secures  building  entry  point 

o  Effectively  utilizes  cover  and  concealment 

o  Moves  along  routes  that  do  not  mask  friendly  suppressive  fires 
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o  Conducts  movement  after  enemy  defensive  fires  are  suppressed 


■  The  squad  reacts  to  contact 
o  Engages  the  enemy 
o  Fixes  the  enemy  to  a  position 
o  Maneuvers  against  the  enemy 
o  Uses  direct  fire  weapons  appropriately 
o  Destroys  the  enemy 

Measured  Decision  Points 

This  scenario  includes  2  measured  decision  points  for  the  trainees  to  address. 

The  standard  for  performance  is  GO  or  NO  GO. 

1 .  General  Orientation-  the  trainees  should  move  in  a  northerly  direction 
towards  the  objective  in  the  simulation. 

2.  Maintain  Security-  the  trainees  should  maintain  security  throughout  the 
simulation.  The  trainees  will  not  have  maintained  appropriate  security 
measures  if  at  any  time  an  enemy  Soldier  approaches  within  5  meters  or 
less  to  the  squad. 

3.  Use  of  Cover-  the  trainees  should  maintain  the  use  of  cover  throughout 
the  simulation.  No  more  than  50%  of  the  trainees  in  the  simulation  should 
be  without  cover  at  any  one  time. 

4.  Takes  Action  Upon  Contact-  upon  contact  the  trainees  will  locate  and 
maneuver  against  the  enemy  until  they  are  eliminated. 

5.  Use  of  Force-  the  trainees  will  eliminate  only  OPFOR. 
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APPENDIX  F 


VISSA  SCENARIO  AUTHORING  & 
OPERATOR  TRAINING 
PRESENTATION 


SCENPRO 


VIRTUAL  SOLDIER  SKILLS  ASSESSMENT  SYSTEM 


Scenario 

Authoring 

And 

Operator 

Training 


&>■'  SCENPRO 

ViSSA  Author  /  Operator  Training  - 

Agenda 

•  Overview  of  ViSSA  System 

•  Phases  of  Operation 

•  Assessment  Rule  Authoring 

•  Pre-Exercise  Setup  Tasks 

•  O/C  Tasks  at  Exercise  Start 

•  O/C  Tasks  during  Exercise 

•  O/C  Tasks  at  Exercise  End 

•  Post-Exercise  Tasks 

•  ViSSA  Object  Model 

•  VB  Script 

•  Advanced  Techniques 

2 
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£  SCENPRO 

ViSSA  Demonstration  Sequence 

•  Description  of  three  applications 

•  Description  of  Training  Exercise  Authoring  Methodology 

•  Use  of  DISAF  (Terrain,  SAFs,  Overlays) 

•  Use  of  ViSSA  Assessment  (Properties,  Terrain, 

Overlays,  Unit  Hierarchy,  Observations,  ECA  Rules) 

•  Exercise 

-  Start  DISAF  and  ViSSA  Application  Suite 

-  Load  scenarios 


1  SCENPRO 

Overview  of  ViSSA  System 

•  The  Virtual  Soldier  Skills  Assessor  is  a  software 
suite  that  works  in  conjunction  with  DIS/HLA 
soldier,  tank,  and  aircraft  simulations. 

•  ViSSA  monitors  soldier  movement  and  behavior  in 
a  virtual  environment  and  compares  it  against  a 
standard  (that  is  defined  by  a  trainer,  commander, 
O/C,  or  Battiemaster). 

•  As  a  soldier  moves  through  a  virtual  world 
executing  a  mission,  ViSSA  assesses  his  or  her  skill 
(against  the  standard)  at  different  points  and 
records  specific  instances  of  good  or  poor  skills 
and/or  decision-making. 

•  ViSSA  can  be  used  at  the  conclusion  of  soldier 
training  to  ensure  that  the  soldier  has  a  high 
degree  of  skill  for  specific  tactics  and  procedures. 
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J  SCENPRO 

ViSSA  Operation 


•  ViSSA  operates  by  monitoring  the  message  traffic  that 
moves  between  separate  simulation  systems. 

•  This  message  traffic  specifies  what  the  solder,  tank,  or 
aircraft  is  doing  -  such  as  moving,  firing  a  weapon,  or 
using  the  radio. 

•  ViSSA  uses  this  message  traffic  in  three  ways: 

-  Observe 

•  Certain  assessments  can  be  made  simply  by  examining  the  contents  of 
a  message 

•  Is  the  soldier  aiming  the  weapon? 

-  Compute 

•  Some  assessments  can  be  made  by  making  small  calculations  between 
different  messages 

•  How  long  did  it  take  the  team  leader  to  report  that  his  team  was  under 
fire? 

-  Infer 

•  It  is  possible  to  infer  certain  mental  processes  by  observing  complex 
soldier  behavior 

•  If  the  team  members  cross  the  square  one  at  a  time,  then  infer  that 
the  team  leader  recognized  that  there  may  be  an  ambush. 
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i  SCENPRO 

ViSSA  Assessment 

•  ViSSA  assesses  soldier  skills  by  evaluating  Rules 
(that  are  created  by  a  trainer,  commander,  O/C,  or 
Battlemaster) 

•  Rules  are  created  in  the  form  of  Event-Condition- 
Action  (ECA) 

-  Events  cause  the  rules  to  be  evaluated 

-  Events  include  the  soldier  moving,  using  a  weapon,  using 
the  radio,  and  getting  wounded 

-  Conditions  are  tests  on  details  of  the  events  or  the  state  of 
the  virtual  world 

-  Conditional  tests  can  be  made  on  the  soldiers  location, 
speed,  orientation,  force,  unit,  role,  etc. 

-  Actions  are  performed  if  the  conditions  are  true 

-  Actions  include  indicating  good  or  poor  skill,  setting  a 
global  variable,  starting  a  timer,  and  ordering  a  SAF  into 
action 


A.  SCENPRO 


Phases  of  Operation 


Before  the  Exercise 

•  ViSSA  Assessment  Module  acts  as  the 
Training  Exercise  Authoring  Tool  to  aid 
the  Trainer,  O/C,  Commander,  or 
Battlemaster  in  creating 
Event/Condition/Action  Rules  to 
monitor  soldier  movement  and 
behavior  to  assess  decision-making 

During  the  Exercise 

•  ViSSA  Assessment  Module  evaluates 
Event/Condition/Action  Rules  against 
soldier  movement  and  behavior 

•  ViSSA  Logger  Module  captures  and 
stores  all  relevant  PDUs  (including 
Mark-Timeline)  during  the  exercise 

After  the  Exercise 

•  ViSSA  Logger  Module  is  used  to  play 
back  exercise  PDU/Logged  data 

•  Logger  Module  GUI  displays  each  of 
the  Mark-Timeline  PDUs  to  focus  AAR 
activities  on  the  key  decision  points  of 
the  mission 
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1  SCENPRO 

ViSSA  -  Before  the  Exercise 


Before  to  the  exercise,  the 
Training  Exercise  Author  uses 
ViSSA  Assessment  and 
DISAF/OneSAF  to  create  a 
scenario.  Products  include: 

*  DISAF/OneSAF  Scenario  File 
-SAFs 

-  Overlays  (Points,  Lines/ 
Routes,  Areas) 

-  Triggers 

*  ViSSA  Scenario  File 

-  Unit  Hierarchy 

-  Observations 

-  E/C/A  Rules 


LANs 


A  SCENPRO 

ViSSA  -  During  the  Exercise 


During  the  exercise,  ViSSA 
Assessment  monitors  PDU 
traffic,  raises  Events, 
evaluates  Conditions,  and 
executes  Actions  (including 
generating  MarkTimeline 
PDUs). 

The  ViSSA  Logger  and 
Audio  Logger  capture  all 
(relevant)  PDU  traffic, 
compute  statistics,  and 
store  data  in  Log  Files. 

ViSSA  Logger  also  provides 
O/C  with  a  method  for 
entering  an  Observation. 


ViSSA 


After  the  Exercise, 
the  ViSSA  Logger  and 
Audio  Logger  piay 
back  the  iogged  PDU 
traffic,  broadcast  this 
traffic  on  the  LAN  for 
use  by  Viewers,  and 
displays  the  ViSSA 
Decision-Making 
Assessment  marks. 


J;  SCENPRO 

After  the  Exercise 


LAN 


iSCENPRO 

Training  Exercise  Authoring 

•  ViSSA  requires  a  standard  against  which  to  assess 
soldier  skills  and  decision-making 

•  Creating  a  "ViSSA  scenario"  is  be  done  prior  to  the 
execution  of  an  exercise  -  at  the  same  time  as  the 
Mission  Briefing,  OPORD,  and  DISAF/OneSAF 
scenario  are  created 

•  The  Training  Exercise  Authoring  Methodology 
(TEAM)  is  a  sequence  of  tasks  that  produce  a 
ViSSA  scenario 

•  The  following  slides  show  the  high-level  steps  of 
the  Methodology 
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J  SCENPRO 

Training  Exercise  Authoring  Methodology 

1.  Identify  Soldier  Skill(s)  being  assessed 

2.  Review  source  of  soldier  knowledge 

a.  Training  Material 

b.  Documentation 

3.  Enumerate  a  set  of  situations  that  cover  the  scope  of 
the  skill 

a.  Just  barely  requires  skill 

b.  Requires  adequate/typical  level  of  skill 

c.  Requires  total  skill  mastery 

d.  Special  cases 

4.  Create  a  scenario,  with  believable  context,  to  stress 
one  or  more  of  the  situations.  This  will  likely  be  in  the 
5-paragraph  OPORD  format. 

5.  Determine  expected  soldier  actions  within  the  scenario 

a.  Under  trained 

b.  Typical 

c.  Expert 


1  SCENPRO 

Training  Exercise  Authoring  Methodology 

6.  Indicate  key  decision  points  within  the  scenario 
relative  to  the  skills  being  assessed 

7.  Identify  observable  actions  that  can  be  used  to 
quantify  the  soldier's  application  of  the  skill  being 
assessed 

8.  Determine,  in  conjunction  with  the  O/C,  how  to  capture 
each  action 

a.  Directly  Observable  with  PDUs 

i.  Motion  of  Soldier  or  SAF  into  or  out  of  Indusion  or  Exdusion  Area 

ii.  Motion  of  Soldier  or  SAF  across  Phase  Line 

iii.  Fires  by  Soldier  or  SAF 

iv.  Wound/Kill  of  Soldier,  Teammate,  or  SAF 

v.  Radio  Transmission  by  Soldier  or  O/C 

vi.  Exercise  Time  (minutes  since  the  start  of  the  exerdse) 

b.  Computable 

i.  Reaction  Time  (time  between  one  action  and  another) 

ii.  Facing  (when  two  entities  CAN  see  each  other) 

iii.  Distance  (between  soldier  and  target  or  rallypoint) 

14 
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Training  Exercise  Authoring  Methodology 

8.  Determine,  in  conjunction  with  the  O/C,  how  to 
capture  each  action  (continued) 

c.  Inferable 

i.  Team  Behavior  (over  march,  covering  fire,  coordinated  assauit) 

ii.  Complex  Routes  (to  avoid  kill  zones) 

iii.  Sophisticated  use  of  Weapons  (smokes  and  flash-bangs) 

9.  Create  a  new  scenario  in  DISAF/OneSAF 

a.  Lay  down  forces  (Friendly,  Neutral,  Enemy) 

b.  Create  Overlays  and  enter  required  points,  lines,  areas,  and 
SAF  triggers  -  annotate  each 

10.  Create  new  scenario  using  ViSSA  Assessment  Module 

a.  Specify  which  forces  are  SAF  and  which  should  be  Assessed 

b.  Specify  specific  Observations  the  O/C  can  make  during  exercise 

c.  For  each  E/C/ A,  include  Socratic  questions  to  support  AAR 

11.  Create  an  OPORD  to  give  to  the  soldier  prior  to 
entering  the  SVS 

12.  Create  instructions  for  the  O/C 


A.  SCENPRO 

Creating  E/C/ A  Rules 

•  Steps  8  and  10c  of  the  Methodology  result  in  the 
creation  of  the  Event/ Condition/Action  Rules 

•  In  order  for  ViSSA  to  "evaluate"  these  Rules,  they 
must  be  provided  in  a  "computer  readable  format" 

•  We  have  selected  Microsoft's  Visual  Basic  Script,  or 
VB  Script,  to  represent  the  E/C/A  Rules. 

•  In  order  for  E/C/A  Rules  to  be  evaluated,  there 
Assessment  system  must  receive  an  Event... such 
as  an  Entity  Movement  or  a  Radio  Transmission. 

•  The  Conditional  part  of  the  E/C/A  Rules  associated 
with  that  event  are  then  evaluated  (using  a  VB 
Script  engine). 

•  Any  time  the  Conditional  part  of  an  E/C/A  Rule 
evaluates  to  TRUE,  the  Action(s)  are  executed. 
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SCENPRO 


Event/Condition/ Action  Rule  Evaluation  Example 


1.  Soldier  I 
moves 
forward 


ViSSA 

Assessment 


street 


3.  ViSSA 
captures  soldier 
Entity-State 
PDU  and  raises 
OnMovement  j 
event ! 


4.ECARules 
associated  with 
OnMovement 
event  are 
1  evaluated 


2.  Entity-  — ' 

v - 

State 

PDU  is 

broadcast 

on  LAN 

LAN 

5.  If  Condition  1 
part  of  any  i 
ECAistruey 


executes 

Action 


sammm 

i  SCENPRO 

“K'i  Yas. 

Event/Condition/ Action  Attributes 

•  Events 

•  Conditions 

•  Actions 

Start  Ex 

Phase 

Set  Phase 

OnMovement 

ElapsecTTime 

Set  Property 

OnFire 

Property 

Capture  Location 

OnDetonate 

OnTransmit 

Health 

Mark  Timeline 

OnObservation 

Point 

Start  Timer 

OnTimer 

Line 

Cancel  Timer 

i  OnTimer  Abort 

Area 

EndEx 

Distance 

Order  SAF 

Orientation 

GnHealthChange 

Posture/Weapon 

OnWeaponCbange 

Force/Unit/Roles 

{  GnPostureChange 

Participation 

)  OnPDUReceived 

Stats 

?  On  Hit*  On  Was  irtri 

Radio  Transmission 

|  OnKill 

Timer  Status 
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Event/ Condition /Action  Example 


•  Description:  If  soldier  moves  into  Kill  Zone  (area 
outside  window  of  hostage  site),  then  Mark-Timeline  and 
cancel  CircleBuildingQuickly  timer 

•  Requirement:  An  area  named  "Kill Zone"  must  have 
been  defined  on  an  Overlay  in  DISAF/OneSAF 

•  Event: 


-  On  Movement 


•  Condition: 

-  Set  KilIZone  =  Areas("KillZone") 

-  If  (KillZone.IsIn(Entity.Location))  Then 

•  Action: 

-  MarkTimeline  (  "Poor",  Entity. Callsign,  "Entered  Kill  Zone") 

-  Timers("CirdeBuildingQuickly").IsEnabled  =  False 
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E/C/ A  Assessment  Rule  Authoring 


There  are  currently  three  ways  to  create  E/C/A 
Assessment  Rules 

-  Use  the  Rule  Templates  in  the  ViSSA  Assessment  Module 
(which  produce  VB  Script) 


•  Fratricide  Rule 

•  Area  Exclusion  Rule 

•  Fire  Reaction  Rule 

•  Observation  Rule 
Copy  VB  Script  from 
supplied  examples 
Write  VB  Script 


jSty  Away  (ran  Window 


&  SpedfiePhete  PmAtuuk 


JWindowKi  Zones 


MR  Questions  _  . ‘  _  _  _  _ _ 

Did  jou  know  (hat  you  were  at  the  enemy's  location? 

Did  you  redee  you  were  standhg  h  front  of  a  widow? 


A  SCENPRO 

Writing  VB  Script 


•  ViSSA  supports  the  Microsoft  standard  Visual  Basic 
Script  language  (VB  Script) 

•  VB  Script  has  access  to  the  ViSSA  Object  Model  -  a 
large  data  structure  storing  key  fact  about  the 
mission  and  the  entities  (soldiers,  SAFs,  tanks, 
etc.) 

•  Most  VB  Script  will  be  associated  with  one  of  the 
Events  (OnMovement,  Start  Ex,  etc.)  and  be  put 
within  that  Event's  handler 

•  There  are  five  powerful  concepts  that  should  be 
considered  when  creating  rules  (see  next  slides) 

-  Phase 

-  Overlays,  Points,  Lines,  and  Areas 

-  Entities 

-  Properties 

-  Timers 
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Phase 

Exercises  are  always  in  one  (and  only  one)  Phase. 

The  Phase  when  every  exercise  starts  is  "Initial." 

The  Phase  when  every  exercise  ends  is  "Final." 
Changing  the  Phase  is  an  Action  available  to  all  EGA 
rules. 

You  can  call  a  Phase  anything  -  although,  like  most 
string  variables  in  VB,  it  is  best  to  avoid  spaces  and 
other  non-alph  a  numeric  characters. 

Typical  Phase  names  are:  Ingress,  Recon,  Stage,  Rally 
OverMarch,  Raid,  CiearBuilding,  Defend,  and  Egress. 
To  test  the  Phase  in  the  Condition  part  of  an  ECA  Rule, 
use  the  VB  Script 

-  If  (Phase  =  "Rally")  Then 

To  change  the  Phase,  use  the  VB  Script 

-  Phase  =  "NewPhaseName" 


A  SCENPRO 

Overlays,  Points,  Lines,  and  Areas 

•  Using  DISAF/OneSAF,  it  is  possible  to  place  an  Overlay 
over  the  battlefield  and  then  drop  Points,  Lines,  and 
Areas  onto  the  Overlay 

•  Each  Overlay  has  a  name,  such  as  ExdusionZones 

•  Points  are  a  single  spot  on  the  battlefield.  Lines  can  be 
one  or  more  segments.  Areas  are  a  series  of  line 
segments  that  form  a  closed  polygon 

•  Each  Point,  Line,  and  Area  has  a  name,  such  as 
RaliyPoint,  PhaseLineAlpha,  and  KilIZone 

•  Points,  Lines,  and  Areas  are  often  used  in  the  Conditional 
part  of  ECA  Rules 

•  ViSSA  has  a  number  of  built-in  functions  using  these 
Overlay  objects 

-  If  (Distance(Ra!lyPoint.Location,  Entity.Location)  <  5.0)  Then 

-  If  (PhaseLineAlpha.HasCrossedLine(ATL.Start,  ATL. Location))  Then 

-  If  (KillZone.IsIn(Tankl. Location))  Then 
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Entities 


•  There  are  many  Entities  in  a  typical  DIS/HLA 
simulation  including 

-  Humans  in  a  Soldier  Visualization  Station  (SVS) 

-  Humans  at  a  desktop  SVS 

-  SAFs  produced  by  DISAF  and/or  OneSAF 

-  Tanks 

-  Aircraft 

-  Furniture  and  other  non-human,  non-vehicle  objects 

•  ViSSA  Assessment  can  reason  about  all  Entities 
except  the  non-human,  non-vehicle  objects 

•  In  VB  Script,  entities  are  accessed  by  their  Callsign 

-  Set  ATLEntity  =  EntitiesfATL") 

-  If  (Entities. Force  <>  "Enemy")  Then 

•  Many  rules  are  restricted  to  particular  Entities  or 
particular  groups  of  Entities  (i.e.  only  Friendly 
forces,  or  only  members  of  Squad  A) 
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Properties 

•  ECA  Rule  Authors  often  must  store  variables 

•  ViSSA  Assessment  allows  Authors  to  store 
variables  as  Properties  on  other  objects 

•  Properties  are  stored  as  the  Variant  data  type 

•  Properties  can  be  used  in  the  Conditional  part  of 
ECA  Rules 

-  If  (ATL.Properties("Wounded")  =  False)  Then 

•  Properties  can  be  set  or  changed  in  the  Action  part 
of  ECA  Rules 

-  ATL.Properties("TimeToReachObjective")  =  25.7 

•  The  following  ViSSA  objects  can  store  properties 

-  ViSSA,  Timers/Timer 

-  Entities/Entity,  EntityRole 

-  Forces/Force,  Units  /  Unit ,  Roles/Role 

-  Overlays/Overlay,  Points/Point,  Lines/Line,  Areas/Area 
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Timers 

•  ViSSA  supports  Timers  -  as  implemented  by 
Windows 

•  Timers  can  be  created  with  a  "timeout"  at  any 
number  of  seconds  from  1  to  64  (with  millisecond 
resolution) 

•  When  a  Timer  is  started  or  re-enabled,  a  separate 
process  will  count  down  the  proper  number  of 
milliseconds  and  then  raise  the  timer  event: 
OnTimer 

•  To  test  a  Timer  in  the  Condition  part  of  an  ECA 
Rule,  use  the  VB  Script 

-  If  (Timers("R5SeekCoverTimer").IsEnabled  =  True)  Then 

•  To  create  a  running  Timer,  use  the  VB  Script 

-  Timers.Add  "  R5SeekCoverTimer  ",  10.0,  True 

•  To  start/reset  a  Timer,  use  the  VB  Script 

-  R5SeekCoverTimer.IsEnabled  =  True 
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Appendix  G 


BattleMaster  Capabilities  Assessment  Questionnaire 
(including  SBL  BattleMaster  responses  from  November  7,  2003) 


Item 

ID 

Evaluation  Statements: 

Strongly 

Agree 

Agree 

Neither 
Agree  or 
Disagree 

Disagree 

Strongly 

Disagree 

Assessment 

'S  '  ;  ■  '  :  S:;  S  ; 

1 

The  ViSSA  training 
exercise  rule  templates 
were  easily  reviewed  for 
evaluation  and 

2 

1 

appropriateness  to  the 
training  objectives  and 
soldier  level  of  experience. 

PBillil  SI  :  .  .  : 

v  " '  '  •  ;  .  "  ' 

•  ?  ‘4  ' 

.  :;s  1 

2 

Using  the  Training 

Scenario  Development 
Methodology  was  helpful 
in  developing  the  training 

3 

exercise. 

•  si? i  -  iS  SS " ;  ■ 

. 

SSSS^IS  ■  S::;  I  y|?SS; 

•  ’  -.f  •;  •  S'' 

3 

The  ViSSA  Assessment 

Tool  allowed  the  training 
exercise  created  to  be  well 
customized  to  the  training 
objectives. 

i 

2 

. . IS  IS  SS  'SSI  I 

;  ’  • .  :  I:  S  S  '  •  '  ’  • 

I-  S  y 

S  S  ;"A  '■  'T >  ;;S 

s  :  .'SS  'ii-iii 

■  K  " 

4 

The  training  exercise  rule 
templates  allowed 
significant  events  to  be 
captured  and  assessed  for 
review  during  AAR. 

2 

1 

: 

'  vi  •  /  % 

* :  •• 

5 

The  ability  to  change 
stealth  viewing  locations 
for  phases  of  the  exercise 
or  significant  events  during 
the  AAR  enhances  the 

2 

i 

ability  to  demonstrate 
good  or  poor  decision¬ 
making. 

iii  ;;:  iiiiii'  is  : i  s  sis: 

■:iS;S;f:  :  :  .V'S  I 
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The  positioning  of  icons  or 
markers  along  a  timeline 
to  identify  the  timing  of 
significant  events  was  very 
helpful  in  conducting  an 
AAR. 

3 
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i-  S ;  • 
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6 

Defining  exercise 
assessment  rules  was 
easily  performed. 

3 
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Item 

ID 

Evaluation  Statements: 

Strongly 

Agree 

Agree 

Neither 
Agree  or 
Disagree 

Disagree 

Strongly 

Disagree 

Exercise  Monitoring 

I 

'  ' v  '  ,, 

7 

Initiation  of  exercise 

3 

recording  was  easily 
performed. 

'■ ' :  :  ■  ’  ,  •  '  ■'  '  " 

>  f ’  V  -  Sj 

8 

Exercise  recording  status 
was  easily  determined. 

3 

'  V::W.  ix'  ' 

-  •  '  ! '  : 

9 

The  manual  insertion  of 
good  &  poor  decision 
points  and  discussion 
point  annotations  was 
easily  performed. 

3 

•  v  |::; . 

'  y  *  I  f$#P%  *  ( 

'  *  /' 

10 

The  status  indicator  that 

2 

i 

audio  recording  was  taking 
place  during  the  exercise 
was  easily  understood. 

f:  "  “  :: 

AAR  Analysis  &  Review 
Support 

;|  ^  ^  % 

”  r  .  U->  ;  %,  -V 

,  &  if  ^  4«s- 

11 

The  recorded  training 
exercise  session  data  for 
AAR  playback  analysis 
was  easily  located  and 
selected. 

3 

/  .  .  .  !.-:S 

•  .  ,  •  :-f 

f§; '  '  . :C:;f . 

'  ' 

12 

The  control  of  exercise 
playback  was  easily 
performed  and 
observations  /  annotations 
were  easily  accessed. 

3 

-  "PM' :M  *:  'i 
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13 

Key  exercise  decision 
points  and  discussion 
points  were  easily 
identified  and 
demonstrated  to  the 
group. 

3 
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14 

Critical  event  capture, 
marking  and  assessment 
was  automatically 
performed  as  specified. 

3 
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15 

Visual  playback  of  the 
simulated  environment 
and  participant  actions 
around  key  exercise 
decision  points  and 
observations  appeared  as 
it  did  in  the  exercise  and 
was  properly  synchronized 
to  the  exercise  timeline. 

3 

W 

P  .  * 

i  |  ; 

16 

Navigation  and  auto¬ 
teleportation  to  desired 
visual  playback  time 
segments  and  viewing 
perspectives  (teleporting 
between  stealth  location 
views,  selected  1st  person 
-  shooter  &  victim  -  views, 
overhead  3rd  person 
views)  were  easily 
performed. 

3 

M 
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■%/;'  4.4  •  4  4. 

ft 
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17 

The  exercise  statistics 
collected  and  displayed 
were  correct  and  useful  to 
the  AAR. 

2 

1 

$ 
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Item 

ID 

Evaluation  Statements: 

Strongly 

Agree 

Agree 

Neither 
Agree  or 
Disagree 

Disagree 

Strongly 

Disagree 
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18 

The  ability  to  repeatedly 
loop  through  decision 
points  /  selected  segments 
of  the  exercise  increased 
AAR  effectiveness. 

2 

'  •; 
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The  variable  playback 
speed  capability  worked 
smoothly  and  made  AAR 
more  effective. 

2 

1 
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21 

The  time  required  to 
process  PDU  packets  to 
create  Exercise  Monitoring 
logs  and  Summary 
Statistics  for  the  AAR  is 
reasonable  and  does  not 
impact  or  delay  the  AAR. 
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AAR  Radio 
Communications 

Playback 

.  Pil  ■ 

•  =  -'i . 

The  training  exercise  data 
for  communications 
playback  was  easily 
located  and  selected. 

2 

| 

i  ■  ';yy 

;  ,  V 

23 

The  communications 
sources  and  recipients 
were  easily  identified  on 
the  user  interface  and 
provided  valuable 
capabilities  to  the 
BattleMaster. 

2 

■  h  ■.  , 

1  ||.  1 

;y : 

y 

24 

The  time  segment  of 
communications  playback 
was  correctly 
synchronized  to  the 
exercise  timeline. 

2 

1  I 

4*$  •  ;v 

|  f 

.#/*!  lf!! 

:  ::  fill: 

25 

The  audio  received  during 
playback  was  of  sufficient 
quality  and  sufficient 
loudness  to  be  easily 
understood  by  the  group. 

2 

jjj 
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26 

The  playback  of 
communications  was 
easily  stopped  and 
restarted. 

2 
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Item 

ID 

Evaluation  Questions: 

:  •/  1  -  ■  .  .  ;y 
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27 

j 

What  additional 
capabilities  are  needed  for 
scenario  rule 
development,  event 
capture,  creation  and 
management? 

1!>  % 

;-i;:  .  .  ::  v 

v?  .v 

-1 

il  7  ^ ; 

.  11  1' 

28 

Which  AAR  functions  need 
enhancement  to  improve 
training  and  mission 
readiness? 

1 .  /  :  . .  .  :  .  ;  m 

.  . . 
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i.:  :  .•  ;  ;• 

G-4 


29 


What  improvements  to  the 
ViSSA  user  interfaces  and 
information  display 
methods  could  greatly 
improve  training 
effectiveness? 

*  Drag  and  Drop 

*  Sort  by  category 

*  Select  All  /  Deselect  All 

•  '  '  '  :  :  ■  v 

l 

What  changes  or  additions 
to  the  Training  Scenario 
Development  Methodology 
would  improve  your  ability 
to  create  highly  effective 
training  exercises? 

ifc  . 
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